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Benefits  and  Costs  of  Distance  Learning: 

A  Perspective  From  the  Distance  Learning 
Literature  since  1995 
Executive  Summary 

Background 

In  1997,  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center- White  Sands 
Missile  Range  (TRAC-WSMR)  published  a  DL  annotated  bibliography  which  covered 
over  a  hundred  DL  reports  in  ten  different  categories,  including  evaluation,  VTT, 
computer-based  training,  cost  effectiveness,  and  guidelines  for  planning  and 
implementing  DL  (Howard  1997).  The  studies  reviewed  covered  the  time  frame  prior  to 
1995.  There  also  have  been  other  literature  reviews  published  since  (Metzko,  Redding 
et  al.  1996;  Russel  1999;  Wisher,  Champagne  et  al.  1999).  This  document  attempts  to 
build  on  the  work  already  done,  and  does  not  intend  to  replicate  already  published 
works.  Specific  areas  reviewed  included  evaluations  of  training  effectiveness  relative  to 
traditional  classroom  instruction,  cost  effectiveness  and  potential  cost  savings,  and 
course  design  or  redesign  requirements  for  DL  courses.  There  was  also  an  attempt  to 
find  relevant  research  on  initial  skill  acquisition  using  DL,  since  initial  skill  learning  is 
different  from  the  learning  required  by  advanced  students. 

Objective 

Objective:  Provide  an  overview  of  the  DL  literature  from  1995  to  the  present  and 
identify  recent  research  findings  relevant  to  specific  issues  and  challenges  faced  by  the 
Army. 

Methodology 

A  literature  search  was  conducted  of  published  studies  and  reports  on  the 
application  of  DL  technologies,  with  the  objective  of  gathering  credible  information 
sources  from  universities,  industry,  professional  and  trade  literature,  and  from  the 
military.  The  emphasis  was  in  reviewing  the  literature  from  1995  to  the  present  to 
identify  the  latest  trends,  issues,  and  research  in  DL.  Additionally,  the  works  included 
were  selected  because  they  are  unique,  well  designed  and  executed,  offer  insight  into 
current  practice  or  developing  DL  trends,  and/or  have  relevance  to  the  Army. 

The  literature  search  focused  on  six  key  subject  areas: 

•  Benefits  of  DL 

•  Cost  of  program  development  and  implementation 

•  Course  design  and  evaluation 


iii 


•  Instructor  and  faculty  issues 

•  Automation  skills  training 

•  Recent  trends  in  the  DL  literature 

Works  for  this  publication  were  selected  to  be  of  particular  interest  and  utility  to 
Army  planners,  training  developers,  and  trainers  and  to  provide  them  with  a  solid 
foundation  in  what  is  currently  known  to  be  sound  principles  and  practices  in  DL  in  this 
dynamic  world  of  technology-based  learning  and  teaching.  Seventy  abstracts  are 
presented  in  appendix  A.  Major  findings  are  summarized  in  the  main  body  of  the 
report. 

Conclusions  and  Recommendations 

The  literature  is  in  general  agreement  that  training  effectiveness  of  DL  courses  can 
be  at  least  equivalent  to  that  of  traditional  classroom  instruction.  Nevertheless,  recent 
literature  cites  a  need  for  tighter  research  controls  to  better  quantify  differences 
between  DL  and  traditional  training  program  outcomes.  The  effectiveness  of  DL  is 
dependent  on  the  quality  of  the  course  materials,  the  skills  of  the  instructor  to  create 
effective  learning  situations,  learning  skills  of  the  student,  and  the  quality  of  the 
equipment,  technical  support,  and  courseware.  With  respect  to  costs,  most  savings 
projections  are  in  the  delivery  of  DL,  not  the  development,  although  this  may  change 
with  evolving  technologies.  A  significant  up-front  investment  is  required  for  DL,  both  in 
technology  and  in  instructional  design  and  instructor  preparation. 

We  are  witnessing  an  evolution  in  the  definition  of  what  constitutes  a  learning 
event.  As  the  learning  event  is  being  redefined,  skills  required  for  learning  are  being 
delineated:  organizing  information,  inquiry  skills,  research  skills,  information  literacy, 
independent  study,  communication  skills,  and  team  problem-solving  skills.  The 
tremendous  advantages  available  from  technology  are  lost  if  the  learning  tools  that 
truly  take  advantage  of  the  new  media  are  not  being  incorporated.  It  is  not  enough  to 
simply  transfer  programs  of  instruction  designed  for  traditional  classroom  delivery  onto 
the  Web.  Courses  need  to  be  redesigned  to  take  into  account  the  inherent  design 
advantages  (or  disadvantages)  of  the  new  technology  over  the  old  technology. 

One  of  the  primary  benefits  of  DL  with  implications  for  Army  training  may  well  be 
the  increased  readiness  of  our  troops  due  to  the  advantages  of  being  able  to  train  troops 
almost  anywhere  in  the  world,  at  anytime,  and  just-in-time.  With  DL,  skills  can  be 
trained,  retrained,  and  refreshed  with  expediency.  Another  important  finding  for  the 
Army  is  that  the  requirements  for  initial  skills  training  vary  from  refresher  training 
requirements.  Multimedia  training  programs  may  need  to  provide  more  aid  to  students 
with  less  initial  knowledge,  including  more  goal-directedness,  feedback  on  activities, 
and  connections  with  previous  knowledge.  The  Army  recruits  soldiers  from  a  diverse 
and  heterogeneous  population  with  varying  levels  of  computer  experience.  Soldier 
training  success  can  be  increased  by  identifying  soldiers  with  little  or  no  computer 
experience  and  providing  them  with  computer  literacy  training  to  get  them  off  to  a 
better  start. 
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Benefits  and  Costs  of  Distance  Learning: 
A  Perspective  From  the  Distance  Learning 
Literature  Since  1995 


Purpose 

The  purpose  of  this  report  is  to  provide  an  overview  of  the  distance  learning  (DL) 
literature  from  1995  to  present  and  to  identify  the  latest  trends  and  issues  in  DL. 

Background 

In  1997,  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center- White  Sands 
Missile  Range  (TRAC-WSMR)  published  a  DL  annotated  bibliography  that  covered  106 
DL  reports  in  10  different  categories,  including  evaluation,  video  teletraining  (VTT), 
computer-based  training,  cost  effectiveness,  and  guidelines  for  planning  and 
implementing  DL  (Howard,  1997).  The  studies  reviewed  covered  the  period  before  1995. 
There  have  also  been  other  literature  reviews  published  since  (Metzko,  Redding,  and 
Fletcher,  1996;  Russell,  1999;  Wisher  et  al.,  1999).  This  document  attempts  to  build  on 
the  work  already  done,  and  does  not  intend  to  replicate  already  published  works. 
Specific  areas  reviewed  included  evaluations  of  training  effectiveness  relative  to 
traditional  classroom  instruction,  cost  effectiveness  and  potential  cost  savings,  and 
course  design  or  redesign  requirements  for  DL  courses.  There  was  also  an  attempt  to 
find  relevant  research  on  initial  skill  acquisition  using  DL,  since  initial  skill  learning  is 
different  from  the  learning  required  by  advanced  students. 

What  is  distance  learning  (DL)?  A  generally  accepted  definition  of  DL  coined  by  Los 
Alamos  National  Laboratory  is  the  "structured  learning  that  takes  place  without  the 
physical  presence  of  the  instructor."  An  extended  definition  from  the  literature  is  that 
"Distance  education  is  planned  learning  that  normally  occurs  in  a  different  place  from 
teaching  and  as  a  result  requires  special  techniques  for  course  design,  special 
instructional  techniques,  special  methods  of  communication  by  electronic  and  other 
technology,  as  well  as  special  organizational  and  administrative  arrangements"  (Wisher 
et  al.,  1999). 

The  impact  of  telecommunications  technologies  is  so  great  that  conceptual  models  of 
teaching  and  of  education  are  coming  into  question  (Syllabus,  November/December 
1999).  Distance  education  today  has  exceeded  the  mere  concerns  of  time  and  space  and 
is  forcing  us  to  question  some  of  the  most  basic  approaches  to  teaching  and  learning 
(Dillon  and  Granger,  1998). 
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Scope 


While  the  concentration  of  effort  was  in  the  review  of  post- 1995  literature,  some 
works  published  before  1995  are  included  where  pertinent  to  the  discussion.  The  focus 
was  on  recent  research  findings  relevant  to  specific  issues  and  challenges  the  Army 
faces  in  trying  to  adapt  rapidly  changing  technology  to  an  equally  rapidly  changing 
Army. 

Methodology 

A  literature  search  was  conducted  of  published  studies  and  reports  on  the 
application  of  DL  technologies,  with  the  objective  of  gathering  credible  information 
sources  from  universities,  industry,  professional  and  trade  literature,  and  from  the 
military.  The  emphasis  was  in  reviewing  the  literature  from  1995  to  the  present  to 
identify  the  latest  trends,  issues,  and  research  in  DL.  Additionally,  the  works  included 
were  selected  because  they  are  unique,  well  designed  and  executed,  offer  insight  into 
current  practice  or  developing  DL  trends,  and/or  have  relevance  to  the  Army. 

The  literature  search  focused  on  six  key  subject  areas: 

•  Benefits  of  DL 

•  Cost  of  program  development  and  implementation 

•  Course  design  and  evaluation 

•  Instructor  and  faculty  issues 

•  Automation  skills  training 

•  Recent  trends  in  the  DL  literature 

Works  for  this  publication  were  selected  to  be  of  particular  interest  and  utility  to 
Army  planners,  training  developers,  and  trainers,  and  to  provide  them  with  a  solid 
foundation  in  what  is  currently  known  to  be  sound  principles  and  practices  in  DL  in  this 
dynamic  world  of  technology-based  learning  and  teaching.  Keyword  searches  in  several 
databases  such  as  Defense  Technical  Information  Center  (DTIC),  Educational  Resources 
Information  Center  (ERIC),  Internet  DL  sites,  libraries,  etc.,  yielded  thousands  of 
information  sources  in  the  rapidly  expanding  DL  field.  From  the  thousands  of 
information  sources  available,  70  studies  were  selected  for  inclusion  in  this  report.  The 
70  abstracts  are  presented  in  appendix  A.  The  abstracts  are  presented  alphabetically 
by  author  and  are  assigned  a  reference  number.  Table  1  provides  a  cross-reference  of 
topics  to  the  numbered  abstracts  contained  in  the  bibliography. 
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Table  1.  Literature  References  by  Topic 


Topic 

Reference  Number 

Benefits  of  DL 

1,  5,  8,  9,  10,  11,  12,  14,  15,  20,  21,  22,  24,  28, 

31,  34,  35,  36,  38,  39,  41,  44,  45,  46,  49,  50,  51, 
52,  53,  57,  58,  59,  60,  61,  62,  65,  66,  67,  69,  70 

Cost  of  Program  Development  and 
Implementation 

4,  5,  6,  7,  13,  22,  23,  24,  28,  29,  31,  34,  35,  39, 

46,  47,  48,  53,  58,  63,  66,  68 

Course  Design  and  Evaluation 

2,  4,  5,  7,  8,  14,  15,  17,  18,  19,  20,  22,  24,  25, 

26,  27,  30,  31,  32,  36,  41,  45,  50,  53,  55,  56 

Instructor  and  Faculty  Issues 

8,  14,  22,  26,  31,  32,  45,  48,  53,  54,  55,  59,  60 

Automation  Skills  Training 

3,  24,  25,  30,  31,  33,  41,  51,  56,  62,  64,  66,  69 

Recent  Trends  in  the  DL  Literature 

16,  37,  39,  40,  42,  43,  49,  53,  54,  55,  61,  69 

The  pace  of  technology-driven  change  is  quickening  and,  as  a  result,  it  becomes 
difficult  to  keep  up  with  and  classify  the  quantities  of  information  now  available  at  one's 
fingertips.  If  one  enters  the  keywords,  distance  learning,  into  an  Internet  search  engine 
such  as  Yahoo,  one  will  get  back  literally  hundreds  of  thousands  of  hits.  Hara  and 
Kling  (Hara  and  Kling,  2000)  define  at  least  five  major  kinds  of  DL  literature:  (1) 
specialized  research  literature,  which  includes  journals  such  as  the  Journal  of 
Asynchronous  Learning  and  the  American  Journal  of  Distance  Education,  research 
monographs,  and  specialized  conferences;  (2)  practitioner  literature  whose  audience  is 
administrators  and  faculty  who  may  be  involved  in  such  courses  (e.g.,  The  Chronicle  of 
Higher  Education)  (3)  instructional  materials  for  students  (course  materials);  (4) 
popular  accounts  (mainstream  press);  and  (5)  marketing  descriptions  of  courses  and 
degree  programs. 

Table  2  contains  a  breakout  of  the  70  DL  literature  abstracts  contained  in  this 
report  by  the  classification  of  DL  literature  described  above  (Hara  and  Kling,  2000). 

The  concentration  of  effort  focused  on  the  specialized  research  and  practitioner 
literature,  with  lesser  attention  given  to  popular  press  and  marketing  materials.  The 
reader  is  directed  to  appendix  A  for  an  overview  of  DL  information  sources,  many  of 
which  are  online. 


Table  2.  References  1 

yy  Literature  Classification 

Classification 

Number  of  References  (n  =  70) 

Specialized  Literature 

56 

Practitioner  Literature 

12 

Instructional  Materials 

0 

Popular  Press 

2 

Marketing  Descriptions 

0 
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Benefits  of  Distance  Learning 

Training  Effectiveness 


The  evidence  from  the  literature  overwhelmingly  supports  the  tenet  that  DL  is  as 
effective  as  traditional  instruction  in  achieving  a  wide  range  of  instructional  objectives. 
The  evidence  comes  from  such  studies  as  the  Institute  for  Defense  Analyses  (IDA) 
review  of  over  300  studies  (Metzko  et  al.,  1996),  Kulik  and  Kuliks'  meta-analysis  of  254 
computer-based  instruction  (CBI)  studies  (Kulik,  1994),  and  Russell's  "No  Significant 
Difference"  phenomenon  as  reported  in  218  studies  (Russell,  1999),  among  many  others. 

However,  two  recent  comprehensive  reviews,  one  military  and  one  academic,  have 
called  into  question  the  way  that  effectiveness  has  been  studied  with  respect  to  DL.  The 
issues  are  both  complex  and  subtle.  DL  is  effective,  but  how  effective  it  is  depends  on 
the  quality  of  the  courseware  (indeed  the  question  might  better  be  what  constitutes 
good  teaching,  whether  face-to-face  or  at  a  distance),  the  learning  skills  and  maturity  of 
the  students,  the  skill  of  the  instructor  teaching  with  this  medium,  the  motivation  of 
students  and  instructors,  as  well  as  the  technology  used.  The  two  studies,  "What's  the 
Difference?  A  Review  of  Contemporary  Research  on  the  Effectiveness  of  Distance 
Learning  in  Higher  Education,"  by  the  Institute  for  Higher  Education  Policy,  April  1999 
(Phipps  and  Merisotis,  1999),  and  "Training  Through  Distance  Learning:  An 
Assessment  of  Research  Findings,"  by  the  United  States  (US)  Army  Research  Institute 
(ARI),  June  1999  (Wisher  et  al.,  1999),  point  out  that  many  evaluations  on  DL  outcomes 
are  anecdotal,  and  that  there  is  a  relative  paucity  of  true,  quality  research  dedicated  to 
explaining  or  predicting  phenomena  related  to  DL.  Implications  of  recent  research  are 
that  technology  cannot  always  replace  the  human  factor  in  education  and  the 
technology  employed  is  secondary  to  pedagogical  factors  such  as  learning  skills,  student 
motivation,  and  the  quality  of  both  the  courseware  and  the  instructor. 

The  report  by  Phipps  and  Merisotis  (Phipps  and  Merisotis,  1999)  has  itself  come 
under  fire.  In  an  article  by  Brown  and  Mack  (Brown  and  Mack,  1999)  their  evaluation 
of  DL  literature  is  described  as  convoluted,  naive,  and  contradictory,  and  their 
expectations  of  the  research  as  unrealistic.  Another  critique,  in  an  editorial  on  the 
report,  further  points  out  that  a  basic  problem  with  critiques  of  individual  studies,  such 
as  the  one  by  Phipps  and  Merisotis,  is  that  they  ignore  the  fact  that  all  of  the  studies 
show  no  significant  difference.  The  piece  of  the  critique  that  is  missing,  one  that  would 
make  the  criticism  even  more  convincing,  is  a  list  of  studies  showing  that  there  is  a 
significant  difference,  and  that  the  difference  is  in  favor  of  classroom  instruction  (Ewell, 
1999).  Another  point  brought  out  in  the  debate  over  significance  studies  in  general  was 
that  the  comparison  of  a  DL  course  to  a  control  group,  a  traditional  classroom,  has 
limits,  among  which  are  the  quality  of  the  traditional  course  of  instruction.  In  the  case 
where  the  traditional  course  is  of  marginal  quality,  all  the  comparison  is  saying  is  that 
DL  performed  as  well  as  a  course  taught  poorly. 
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A  key  issue  with  respect  to  empirical  studies  of  skills  training  in  general  was 
documented  in  the  third  report  of  the  National  Academy  of  Sciences'  select  Committee 
on  Techniques  for  the  Enhancement  of  Human  Performance  (CTEHP),  which  is  the 
tendency  of  trainees  and  trainers  to  inadvertently  fool  themselves  and  each  other  about 
what  and  how  much  has  been  learned  and  trained  (Druckman  and  Bjork,  1994).  This 
unique  14-member  committee  was  formed  in  1985  to  evaluate  the  potential  of  a  number 
of  innovations  and  specific  techniques  that  were  claimed  to  have  enhanced  human 
performance  to  a  significant  degree.  The  committee  found  that  the  traditional  climate 
of  military  training,  including  pressures  for  efficiency  and  intolerance  of  errors,  may 
reinforce  such  tendencies  to  over-measure  success.  The  purpose  of  training,  after  all,  is 
post-training  performance.  Their  analysis  indicated  that  training  conditions  that 
enhance  skill  acquisition  might  retard  skill  retention  and  transfer  to  job  performance. 

Indeed,  many  who  study  these  issues  in  universities  believe  that  new  paradigms  of 
DL  and  online  instruction  call  for  new  paradigms  of  evaluation  (Faculty,  1999).  New 
research  has  set  forth  sets  of  guiding  principles  for  DL  teaching  and  learning,  which  can 
be  viewed  in  light  of  determining  key  elements  for  evaluation  (Association  of  Distance 
Education  Colleges  (ADEC),  2000;  Harasim,  Hiltz,  Teles,  and  Turoff,  1995;  Indiana 
Higher  Education  Telecommunication  System  (IHETS),  2000).  In  addition  to  the 
question  on  whether  learning  competence  is  equal  to  or  superior  to  that  of  a  traditional 
classroom,  DL  related  issues  include  the  following:  is  the  teaching  style  innovative,  are 
students  engaged  in  the  material,  is  there  interaction  between  professors  and  their 
students  and  between  the  students  themselves,  is  access  to  technical  support  readily 
available,  and  are  the  signs  of  academic  maturity  evident,  e.g.,  are  the  students 
thinking  critically,  and  is  there  a  desire  for  life-long  learning  being  fostered  in  them? 

Other  Benefits 

Benefits  that  can  accrue  from  DL  also  include  improved  access  for  students. 
Individual  motivation  and  access  may  be  improved  by  the  distance  education  course 
because  it  either:  (1)  appeals  to  a  broader  range  of  individual  learning  styles  than  does 
classroom  instruction  or  (2)  it  makes  it  easier  for  individuals  to  participate  in  learning 
activities.  Another  benefit  noted  from  academic  publications  is  a  sense  of  institutional 
renewal  for  both  the  university  and  the  faculty,  such  as  new  energy,  fresh  ideas,  and  a 
renewed  sense  of  discovery  and  innovation.  It  is  demonstrated  in  its  potential  to 
motivate  the  institution  and  the  faculty  to  rethink  and  reform  the  curriculum  and  the 
way  courses  are  offered.  Faculty  enthusiasm  provides  the  basis  for  institutional  change 
and  is  acknowledged  to  be  a  very  important  aspect  for  adoption  of  DL.  Further, 
improved  exam  scores  and  course  completion  rates  ultimately  translate  into  improved 
performance  in  other  courses,  a  reduced  rate  of  withdrawal  from  the  university,  and 
higher  degree  completion  rates.  There  is  also  a  potential  social  benefit  of  an  increased 
proportion  of  the  population  that  is  better  educated  (Jewett,  1999). 

Military  studies  have  also  identified  training  benefits  associated  with  DL  courses 
(Howard,  1997;  Metzko  et  al.,  1996;  Westfall,  1998).  "The  economic  value  of  increased 
training  accessibility  and  the  resulting  increased  readiness  available  from  more 
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comprehensive  use  of  DL  technologies  ...  may  prove  to  be  the  primary  payoff  from 
systematic  adoption  of  DL  technologies  for  RC  training"  (Metzko  et  al.,  1996). 

The  benefits  of  DL  for  military  training  include: 

•  Can  be  just  as  effective  as  resident  instruction 

•  Offers  "just-in-time"  scheduling 

•  Saves  travel  time  and  per  diem  costs 

•  Offers  the  ability  to  reach  soldiers  who  could  not  otherwise  be  reached 

•  Can  reduce  backlog  of  training  requirements 

•  Can  offer  enhanced  instruction,  remediation  techniques,  and  real-time  methods 
of  communication  and  feedback 

•  Can  make  correspondence  programs  interactive 

•  Encourages  life-long  learning 

Cost  of  Program  Development  and  Implementation 

A  number  of  recent  university  studies  have  evaluated  the  costs  of  DL  courses 
relative  to  traditional  delivery  modes.  Major  highlights  from  these  cost  studies  are 
summarized  below. 

The  University  of  California  Special  Project,  which  evaluated  the  benefits  and  costs 
of  10  different  case  studies  in  distributed  learning,  found  that  economies  of  scale  could 
exist  at  the  course  level  (Jewett,  1999).  Average  costs  decline  as  the  fixed  costs  are 
spread  over  larger  course  enrollments.  The  total  costs  of  distance  education  rise  more 
slowly  than  those  of  classroom  instruction.  This  is  because  the  incremental  costs  of 
adding  students  to  a  distance  course  are  less  than  those  of  a  classroom  course,  and  the 
two  cost  functions  ultimately  converge,  and  at  some  level  of  enrollment,  they  intersect. 
Their  cost  model  showed  the  crossover  point  in  the  range  of  150-300  average  annual 
course  enrollments  with  an  assumed  course  production  cost  of  $200,000,  a  5-year  useful 
life  for  the  instructional  materials,  and  interaction/evaluation  costs  in  the  range  of  $150 
to  $250  per  student.  Doubling  course  production  costs  moves  the  crossover  point  closer 
to  400  enrollments.  Course  sharing,  in  which  two  or  more  campuses  share  the  course 
production  costs  and  all  list  the  shared  course  as  its  own  (each  campus  continues  to 
incur  the  student  interaction  and  evaluation  costs  associated  with  its  own  students), 
provides  the  potential  for  total  course  enrollments  sufficient  to  obtain  the  benefits  of 
economies  of  scale  (Jewett,  2000). 

A  note  of  caution  with  regard  to  economies  of  scale  is  also  found  in  recent  DL 
literature.  The  large,  industrial-sized  economies  of  scale  envisioned  by  some  university 
administrators  may  not  be  realistic  according  to  the  findings  of  an  18-month  long 
academic  review  of  the  value  of  DL  by  an  interdisciplinary  faculty  group  (University  of 
Illinois  Faculty  Seminar)  that  convened  to  study  online  teaching  and  learning  (Faculty, 
1999).  Because  high  quality  online  teaching  is  time  and  labor  intensive,  it  is  not  likely 
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to  be  the  income  source  envisioned  by  some  university  administrators.  The  scenario 
once  envisioned  of  hundreds  of  students  enrolling  in  a  well  developed,  essentially 
instructor-free  online  course  does  not  appear  realistic.  With  some  exceptions,  the 
successful  online  course  features  low  student  to  faculty  ratios.  The  exceptions  involve 
extraordinary  amounts  of  the  professor's  time.  Besides  the  initial  investment  in 
technology,  technical  support  for  professors  and  students  and  maintenance  of  hardware 
and  software  are  very  expensive.  Online  instruction  is  not  likely  to  cost  significantly 
less  than  traditional  instruction  unless  the  number  of  students  taught  is  large  or  the 
materials  do  not  require  frequent  updates. 

Six  recent  university  studies  commissioned  by  the  Alfred  P.  Sloan  Foundation 
indicated  that  most  of  the  universities  that  participated  in  the  studies  are  near  the 
break-even  point  with  their  DL  programs  (Bishop  and  SchWeber,  2000;  Geith  and 
Vignare,  2000;  Hislop,  2000;  Miller,  2000;  Sachs,  2000;  Wentling,  2000).  The  studies 
also  found  that  defining  costs  is  a  difficult  process,  that  university  accounting 
procedures  were  not  always  well  suited  to  a  case-study  approach,  and  that  small 
variations  in  class  size  could  affect  profitability.  While  costs  of  technology  and  technical 
support  required  for  DL  programs  are  greater  than  traditional  programs,  the  physical 
infrastructure  that  is  needed  to  support  traditional  programs  will  ultimately  make  the 
difference  in  profitability.  Unlike  technology,  the  cost  of  land,  buildings,  and  utilities  do 
not  go  down  over  time.  There  is  potential  for  online  courses  to  be  20  percent  cheaper 
than  traditional  courses  due  to  the  costs  of  the  physical  infrastructure  required  for 
traditional  delivery. 

Technology  is  expected  to  go  down  in  price,  or  rather,  computational  power  per  unit 
cost  will  increase,  and  at  an  exponential  rate  (Kurzweil,  1999).  Ray  Kurzweil,  an 
award-winning  physicist,  computer  scientist,  and  author,  projects  that  by  2019,  a  $1,000 
personal  computer  will  have  the  computational  capability  of  the  human  brain  (20 
million  billion  calculations  per  second)  and  computers  will  be  largely  invisible  and 
embedded  everywhere.  The  growth  in  computational  power  is  irrespective  of  the 
particular  computer  hardware  technology,  as  new  computing  technologies  are  being 
developed  that  will  replace  the  current  chip-based  systems.  According  to  Kurzweil, 
people  will  soon  be  routinely  using  three-dimensional  displays  built  into  their  glasses  or 
contact  lenses.  Most  learning  will  be  accomplished  using  intelligent  software-based 
simulated  teachers.  To  the  extent  that  teaching  is  done  by  human  teachers,  the 
teachers  will  not  be  in  the  local  vicinity  of  the  students  and  will  play  a  mentoring  and 
counseling  role  as  opposed  to  being  the  sources  of  learning  and  knowledge.  Reading 
books,  newspapers,  and  other  Web  documents,  listening  to  music,  watching  movies  or 
television,  and  entering  virtual  environments  will  all  be  done  through  a  communications 
web  that  does  not  require  any  equipment,  devices  or  objects  that  are  not  worn  or 
implanted.  These  projections  and  others  like  them  indicate  the  likely  development  over 
the  next  10  to  20  years  of  new  technologies  that  we  have  not  even  visualized  today. 

Their  impact  is  understandably  unknown  at  this  time,  but  potentially  revolutionary. 
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Course  Design  and  Evaluation 


The  impact  of  telecommunications  technologies  is  so  great  that  conceptual  models  of 
education  are  coming  into  question.  The  term  "paradigm  shift,"  first  introduced  in  the 
early  1960s  by  science  historian  Thomas  Kuhn  to  describe  the  discontinuous  nature  of 
the  progress  of  science,  is  being  applied  more  and  more  frequently  to  the  rapidly 
growing  movement  toward  digital  instruction.  According  to  Kuhn,  scientific  knowledge 
is  organized  within  paradigms  or  conceptual  systems.  Scientists  hang  on  to  familiar 
frames  and  concepts  for  as  long  as  they  can,  only  allowing  a  paradigm  shift  to  take  place 
long  after  it  should  have  become  plain  that  the  old  concepts  do  not  suffice.  This 
paradigm  shift  is  similar  to  how  all  learning  takes  place.  It  is  not  a  smooth,  gradual 
process,  but  can  be  best  illustrated  as  a  series  of  jumps  from  one  plateau  to  the  next. 
With  the  current  technology  infusion  within  teaching  and  learning,  the  very  notion  of  a 
course  or  a  classroom  are  issues  of  current  debate  (Dillon  and  Granger,  1998).  What  is 
"the  class"  for?  This  is  the  question  being  posed  by  educators.  New  technologies  are 
making  information  access  and  learning  possible  in  ways  that  previously  did  not  exist. 
Learning  itself  is  becoming  the  significant  activity,  which  coincides  with  increasing 
learning  needs  in  an  information  society.  A  new  "learning  space,"  in  which  the  total 
context,  or  set  of  conditions,  within  which  learning  takes  place,  is  being  defined 
(Raschke,  1999). 

Current  issues  in  the  course  design  and  evaluation  literature  include  the  following: 

Student  to  Instructor  Ratios 

Online  class  size  may  have  an  inherent  limitation,  determined  by  the  amount  of 
effort  required  to  form  "a  community  of  learners."  The  instructor's  challenge  is  to  create 
appropriate  conditions  for  a  group-learning  environment.  To  be  really  effective,  there 
must  be  a  great  deal  of  interaction  between  the  students  themselves.  In  online  courses, 
the  workload  of  faculty  is  directly  related  to  the  number  of  students.  Students  have 
expressed  frustration  when  there  is  a  lack  of  immediate  feedback  from  the  instructor, 
from  ambiguous  instructions,  or  from  ineffective  communication  (Hara  and  Kling,  2000). 

Small  class  sizes  and  the  development  of  a  sense  of  community  may  be  especially 
important  in  light  of  recent  research  on  the  impacts  of  heavy  Internet  usage  on  family 
and  social  networks.  The  study,  published  in  the  American  Psychologist  in  September 
1998  (Kraut  et  al.,  1998)  used  longitudinal  data  to  examine  the  impacts  of  the  Internet 
on  73  households  during  their  first  2  years  online.  Greater  use  of  the  Internet  was 
found  to  be  associated  with  declining  communication  with  family  members,  declining 
size  of  social  circles,  and  increasing  depression  and  loneliness. 

Network  and  distance  technology  tools  can  be  used  successfully  to  enhance  student 
success  rates  in  large  classes  (Kashy,  Thoennessen,  Tsai,  Davis,  and  Wolfe).  In  large 
classes,  technology  tools  can  be  used  for  computer-assisted  grading  of  assignments, 
quizzes,  and  examinations,  permitting  a  reallocation  of  instructors'  and  teaching 
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assistants'  time  away  from  repetitive  jobs  such  as  grading  and  record  keeping  to  tasks 
more  directly  related  to  student  achievement. 

Student  Interaction 


In  DL  classes,  student  interaction  has  become  an  important  measure  for  assessment 
and  evaluation  (Chickering  and  Ehrmann,  1997;  Hara  and  Kling,  2000;  (IHETS),  2000; 
Policy,  2000).  During  the  1980s,  educators  came  up  with  the  pivotal  seven  principles  for 
good  undergraduate  education,  which  have  been  updated  recently  to  reflect  technology 
advances  (Chickering  and  Ehrmann,  1997).  Two  of  these  principles  deal  directly  with 
student  interaction  among  themselves  and  with  faculty.  The  first  of  the  seven 
principles  is  that  "Good  Practice"  encourages  contacts  between  students  and  faculty. 

The  update  concludes  that  new  communication  technologies  can  increase  access  to 
faculty  members  and  provide  for  joint  problem  solving  and  shared  learning  and  can 
strengthen  faculty  interactions  with  all  students,  but  especially  with  shy  students,  or 
non-English  native  speakers.  The  second  principle  is  that  "Good  Practice"  develops 
reciprocity  and  cooperation  among  students.  Learning  is  enhanced  when  it  is  more  of  a 
team  effort  than  a  solo  race.  The  update  notes  that  the  increased  opportunities  for 
interaction  noted  above  with  faculty  apply  equally  to  communication  with  fellow 
students.  Study  groups,  group  problem  solving,  collaborative  learning,  and  feedback 
can  all  be  strengthened  through  communication  tools  such  as  e-mail. 

Student  interaction  is  also  one  of  the  24  assessment  benchmarks  for  distance 
education  recently  delineated  by  the  Institute  for  Higher  Education  Policy  (Policy, 

2000).  "Student  interaction  with  faculty  and  other  students  is  an  essential 
characteristic"  of  a  high  quality  Internet-based  course. 

Target  Audience  Assessment 

Multimedia  training  programs  may  need  to  provide  more  aid  to  students  with  less 
initial  knowledge  (Lee,  Gillan,  and  Harrison,  1996).  This  laboratory  assessment  found 
that  the  multimedia  software  package  was  less  beneficial  for  students  with  less  initial 
knowledge  in  the  topic  and  that  for  training  such  students,  multimedia  software  may 
need  to  include  more  goal-directedness,  feedback  on  activities,  and  connections  with 
previous  knowledge. 

The  need  for  motivation,  a  key  component  of  a  student's  involvement  in  learning,  is 
a  function  of  a  student's  age  and  background,  intellectual  capacity,  and  psychological 
makeup.  Young  students  may  require  more  frequent  student-faculty  contact  in  and  out 
of  class  than  older,  more  mature  students.  This  is  the  primary  reason  given  for 
advocating  that  young  students  experience  the  traditional  university  on-site  educational 
delivery,  at  least  in  part,  rather  than  going  wholly  DL  for  their  undergraduate  degree 
programs.  The  role  of  attentiveness  given  by  the  professor  to  his  students  imparts  the 
"human  touch,"  providing  the  spark  for  the  process  of  socialization  and  competence 
building  (Faculty,  1999). 
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Another  issue  related  to  target  audience  assessment  is  the  different  computer 
backgrounds  of  individuals  entering  computer-mediated  instruction  and/or  a  DL  course. 
Several  recent  studies  have  documented  the  heterogeneous  computer  backgrounds  of 
the  US  population  (Bachman,  Segal,  Freedman-Doan,  and  O'Malley,  2000;  Gladieux  and 
Swail,  1999;  Hoffman  and  Novak,  1998).  Although  computer  literacy  may  be  on  the 
rise,  mixing  computer-savvy  individuals  with  those  who  have  little  to  no  experience  can 
lead  to  less  than  successful  results.  Solutions  mentioned  in  the  literature  include 
requiring  students  to  enter  the  class  with  a  basic  literacy  of  computer  skills  provided  by 
paid  classes  at  a  local  community  college  or  training  center  for  example,  or  to  extend  the 
training  program  to  include  these  skills.  Prerequisite  courses  in  general  computing  was 
also  a  suggested  solution.  Another  suggestion  from  the  literature  was  to  group  students 
in  small  teams  with  at  least  one  skilled  user  on  each  team  to  help  the  others  (Compton, 
1999). 

Some  major  universities  are  requiring  that  students  who  take  distance  education 
courses  take  an  "online  orientation  course."  Also  emerging  are  some  new  internal 
consulting  groups  and  workshops  for  prospective  DL  instructors.  University 
administrators  are  starting  to  recognize  online  courses  as  a  complex  instructional 
engagement  requiring  new  materials  and  new  behaviors  on  both  the  part  of  the 
students  and  the  instructors  (Hara  and  Kling,  2000). 

Communication 

The  teaching  paradigm  is  changing  for  online  instruction,  away  from  the  traditional 
lecture  format.  The  online  environment  is  essentially  a  space  for  written  interaction  at 
the  present  time,  and  writing  is  yet  another  fundamental  medium  of  expression  with  its 
own  properties  and  powers.  In  the  hands  of  professors  who  know  what  they  are  doing, 
online  instruction  can  be  superior  to  face-to-face  instruction  (Harasim  et  al.,  1995).  The 
ability  to  sit  and  think  through  questions  and  comments  generally  result  in  students' 
contributing  material  that  is  much  better  than  something  they  would  say  off  the  top  of 
their  heads  in  a  face-to-face  class.  However,  those  students  without  solid  writing  and/or 
typing  skills  are  at  a  distinct  disadvantage. 

On  the  other  hand,  brief  and  incomplete  responses  often  lead  to  miscommunication. 
The  study  performed  by  Hara  and  Kling  (Hara  and  Kling,  2000)  documents  students' 
communication  problems  and  other  sources  of  distress  while  taking  an  online  course. 
When  the  main  form  of  communication  is  in  writing,  complexities  and  ambiguities  arise 
due  to  the  nature  of  text-based  media.  The  students  must  anticipate  the  level  of  detail 
and  phrasing  that  will  be  sufficiently  helpful  to  others.  In  this  study,  students  were 
unsure  of  the  appropriateness  of  certain  kinds  of  e-mail,  for  example,  an  e-mail  that 
represents  the  nodded  heads  of  a  face-to-face  group;  does  this  type  of  message  help  to 
confirm  one's  understanding  of  the  subject  matter  or  does  it  simply  result  in  yet  more  e- 
mail  glut?  The  authors  note  that  key  issues  regarding  reliable  and  well-timed 
communication  need  to  be  negotiated  within  a  class  so  that  students  and  instructor's 
expectations  regarding  e-mail,  feedback,  technical  support,  etc.  can  be  aligned,  thereby 
reducing  frustration  and  anxiety. 
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The  instructor  has  an  important  role  in  moderating  the  online  course  interaction. 
The  role  of  the  instructor  as  online  moderator  is  required  to  compensate  for  the  absence 
of  physical  cues  found  in  the  traditional  classroom.  Students  must  be  informed  as  to  the 
exact  nature  of  the  online  conference,  whether  it  is  a  class,  or  a  meeting,  or  a  support 
group,  for  example.  Communications  must  be  monitored  to  assure  that  all  students  are 
participating  and,  finally,  the  instructor-moderator  must  provide  "weaving  comments" 
which  summarize  the  state  of  the  discussion  and  provide  a  sense  of  accomplishment  and 
direction  (Faculty,  1999).  New  technologies  such  as  video  streaming  will  render 
obsolete  many  of  the  problems  encountered  with  the  text-based  medium  of  current 
asynchronous  online  course  offerings,  while  no  doubt  creating  new  and  equally  complex 
pedagogical  issues. 

Course  Design 

Course  design  requirements  are  tied  to  the  media  used  for  presentation.  A  good 
discussion  of  different  media  presentation  features  with  respect  to  learning  tasks  and 
developing  skills  can  be  found  in  A.W.  Bates  "Technology,  Open  Learning,  and  Distance 
Education"  (Bates,  1995).  Bates  warns  that  it  is  common  for  educators  and  media 
specialists  to  carry  over  modes  of  design  associated  with  an  "old"  technology  to  a  newer 
technology,  even  though  the  new  technology  may  have  inherent  design  advantages  (or 
disadvantages)  over  the  old  technology.  Thus,  professors  often  use  television,  for 
example,  to  relay  lectures,  rather  than  designing  their  lecture  to  maximize  televisions' 
presentational  characteristics.  Another  practice,  often  referred  to  as  shovelware,  is  for 
faculty  to  simply  put  their  syllabi  on  the  Web  and  then  transfer  over  their  lectures  as 
text.  Again,  with  this  approach,  the  learning  tools  that  truly  take  advantage  of  the  Web 
media  are  not  being  incorporated  (Davis,  1999).  The  tremendous  opportunities 
available  from  technology  are  lost  if  its  advantages  are  not  fully  exploited. 

Using  video  as  a  broadcast  medium  for  DL  continues  to  be  a  very  viable  option  for 
many  universities  and  colleges.  At  the  Public  Broadcasting  System  (PBS),  video  is  a 
key  instructional  medium  due  to  the  high  level  of  motivation  and  understanding  it  can 
engender,  according  to  the  director  of  learning  innovations  for  the  PBS  Adult  Learning 
Service  (ALS)  (Davis,  2000).  PBS,  in  collaboration  with  Intel,  is  also  becoming  actively 
involved  in  "enhanced  television,"  a  recently  developed  distribution  method  that  enables 
supplemental  digital  information  to  be  broadcast  along  with  a  digital  video  signal  that 
can  be  captured  by  a  computer  for  nonlinear  access.  Over  the  past  5  years,  the  biggest 
change  in  communications  around  the  world  has  been  the  emergence  of  mature  digital 
technologies,  as  nearly  all  communications  move  from  analog  to  digital.  "We  are  just 
beginning  to  understand  their  potential  for  education,"  according  to  Davis  (Davis,  2000). 

New  Guidelines  for  Course  Evaluation 

New  benchmarks  for  evaluating  distance  education  courses  have  been  developed 
over  the  last  year.  A  study  undertaken  by  the  Institute  for  Higher  Education  Policy 
(Policy,  2000)  culminated  in  a  list  of  24  benchmarks  to  ensure  quality  Internet-based 
courses.  The  new  benchmarks  reflect  the  new  requirements  of  teaching  and  learning  at 
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a  distance.  Some  of  the  24  benchmarks  listed  include:  having  a  documented  technology 
plan;  setting  minimum  standards  for  course  development,  design  and  delivery;  ensuring 
that  students  engage  themselves  in  analysis,  synthesis,  and  evaluation;  and  student 
interaction  with  faculty  and  students  and  the  measurement  of  this  interaction. 

Students  also  need  to  have  access  to  sufficient  library  resources  that  may  include  a 
"virtual  library"  accessible  through  the  World  Wide  Web  (WWW). 

Other  benchmarks  refer  to  faculty  support  requirements  in  the  forms  of  technical 
assistance  in  course  development,  help  in  transitioning  from  classroom  to  online 
instruction,  and  instructor  training  and  assistance,  including  peer  mentoring, 
throughout  the  course. 

Instructor  and  Faculty  Issues 

Nowhere  does  the  two-sided,  pi'omise  or  peril,  nature  of  distance  education  come  into 
play  more  directly  than  with  respect  to  instructors  and  faculty.  Their  issues  and 
concerns  cover  topics  ranging  from  quality  of  education,  defining  what  comprises  good 
teaching,  retraining  to  learn  the  new  skills  and  technologies,  to  copyright  and 
intellectual  property  issues. 

Adverse  Faculty  Reactions  to  DL  Programs 

Early  ventures  to  create  the  California  Virtual  University,  a  consortium  of  public 
and  independent  colleges,  and  the  California  Educational  Technology  Initiative  (CETI), 
which  was  formed  to  facilitate  business  transactions  between  California  State 
University  (CSU)  and  a  number  of  computer  firms,  were  defeated  due  to  concerned 
faculty  resistance  and  lack  of  fundamental  support.  Proposals  for  radical  retooling  of 
universities  have  provoked  faculty  hostility.  Faculty  argue  that  administrators  are 
mainly  concerned  with  the  money-making  potential  of  distance  education,  and  that 
when  administrators  are  solely  in  charge  of  implementing  technology  in  the  classroom, 
there  is  the  risk  of  "technology  driving  pedagogy"  instead  of  "pedagogy  driving 
technology"  (Faculty,  1999). 

Good  Teaching  is  Good  Teaching 

Another  issue  highlighted  in  recent  DL  literature  is  how  to  translate  good  teaching 
in  the  classroom  to  good  teaching  online  (Chickering  and  Ehrmann,  1997;  Faculty,  1999; 
Policy,  2000;  Ragan,  1998).  Seven  principles  of  good  teaching  developed  by  the 
American  Association  for  Higher  Education  in  1987  have  been  updated  for  online 
teaching  in  1997  by  taking  each  of  the  original  seven  principles  and  applying  them  to 
new  learning  technologies  (Chickering  and  Ehrmann,  1997). 

Faculty  Roles 

As  universities  rapidly  develop  courses  for  delivery  via  the  Internet,  what  will  be 
the  impact  on  faculty  roles  and  responsibilities?  One  impact  discussed  in  the  literature 
is  the  division  of  labor  between  those  who  develop  and  prepare  content  and  those  who 
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teach  it  (Moore,  2000).  It  is  unlikely  that  universities  will  be  able  to  compete  in  the 
global  market  of  distance  education  without  changing  its  organizational  structures, 
including  the  roles  of  its  faculty.  Institutions  that  deliver  distance  education  courses  as 
an  "add-on"  to  traditional  teaching  will  eventually  be  overtaken  by  the  higher  quality  of 
design  and  facilitation  provided  by  distance  education  specialists.  There  may  be  fewer 
subject  specialists  who  design  and  prepare  the  course  content,  with  the  majority  of  the 
faculty  functioning  as  supporting  facilitators.  Institutions  will  have  to  invest  in  areas 
where  they  have  a  comparative  advantage  and  close  out  areas  in  which  they  do  not  have 
the  expertise.  Paying  famous  professors  to  design  or  give  their  seal  of  approval  for  a 
course  will  become  a  one-time  cost.  Online  teaching  can  be  done  by  teaching  assistants 
and  qualified  facilitators  anywhere  in  the  world. 

The  Army  can  take  advantage  of  the  economies  of  scale  available  in  online  teaching, 
such  as  savings  in  travel  and  per  diem,  without  having  to  deal  to  a  great  extent  with 
faculty  role  issues,  because  the  Army  has  traditionally  separated  the  course 
design/development  and  teaching  roles.  The  Army  may  also  benefit  from  DL  in  that  all 
soldiers  requiring  training  in  a  particular  subject  area  could  feasibly  be  exposed  to 
training  designed  by  the  very  best  experts  in  that  field. 

Automation  Skills  Training  Summary 

Computer  Literacy 

Computer  literacy  is  so  important  that  it  needs  to  become  an  explicit  goal  of  the 
armed  services,  advised  a  group  of  Silicon  Valley  chief  executive  officers  (CEOs)  during 
a  Marine  Corps  Advanced  Warfighting  Experiment  (AWE)  that  paired  10  generals  and 
5  admirals  with  15  top  business  leaders  from  Silicon  Valley.  The  primary  purpose  of  the 
wargame  was  for  the  military  to  obtain  the  advice  and  business  perspective  of  the 
leading  commercial  information  technology  (IT)  corporations  with  respect  to  IT 
investments.  Advice  from  the  business  executives  included  reexamining  the  disparity 
between  IT  investment  in  garrison  (41  percent  of  total  budget)  versus  the  tactical  level 
(17  percent)  and  investing  more  into  internal  Marine  training  to  advance  computer 
literacy,  adding  that  they  believed  the  entire  force  should  be  trained  to  be  computer 
literate  by  2010  (West,  1999). 

Computer  literacy,  especially  the  ability  for  leaders  and  staffs  to  use  digital 
command  and  control  (C2)  systems,  has  become  a  decisive  battlefield  skill.  What  is  not 
clear  is  how  best  to  achieve  it.  The  results  of  an  Army  training  experiment  at  the  US 
Army  Command  and  General  Staff  College  (CGSC)  indicated  that  both  computer 
performance  skills  and  brigade-level  tactical  competency  could  be  achieved  using  the 
same  amount  of  time  previously  devoted  to  tactics  training  alone.  The  existing  tactics 
course  was  extensively  redesigned  to  integrate  a  previously  separate  40-hour  computer 
operator  course.  The  new  course  design  was  based  on  the  constructivist  approach  to 
instruction,  which  places  the  learner  in  an  active  role  in  which  knowledge  is  discovered 
in  interaction  with  the  environment.  Using  the  constructivist  model,  the  classroom 
environment  must  be  as  close  to  the  environment  that  the  task  is  performed  under  as  is 
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practical.  The  learner  must  become  engaged  in  the  learning  process,  interpreting 
experiences  and  constructing  his/her  own  guide  to  how  knowledge  can  be  used.  The 
instructor  intervenes  only  when  the  students  reach  a  point  of  no  progress.  The  after 
action  review  (AAR)  is  a  key  component  of  the  approach  (Ross  and  Yoder,  1999). 

Since  the  Army  began  experimenting  with  digitization  in  1994,  researchers  have 
learned  that  how  information  is  conveyed  is  as  important  as  what  information  is 
conveyed.  Studies  revealed  that  information  needed  to  be  displayed  on  large  screens  at 
the  battlefield  headquarters.  Commanders  made  decisions  on  visually  seeing  how 
things  are  presented.  Second,  data  from  different  systems  needed  to  be  integrated  to 
make  it  useful  for  commanders,  e.g.,  staff  officers  had  to  be  able  to  synchronize  data 
arriving  through  intelligence  and  logistics  channels  with  data  arriving  through  combat 
system  channels.  In  addition,  soldiers  have  to  develop  a  higher-than-average  level  of 
computer  literacy  to  use  and  maintain  the  digitized  systems  effectively.  There  is  a 
constant  need  for  training  and  for  maintenance  of  this  highly  technical  infrastructure 
(Peters,  1999). 

Initial  Skills  Training 

A  recent  military  pilot  study  assessed  the  training  effectiveness  of  interactive 
multimedia  instruction  (IMI)  training  packages  for  training  soldiers  on  complex 
automated  systems  (George,  Bretl,  and  Jackson,  2001).  The  study  found  that  soldiers 
who  had  difficulty  learning  these  complex  automated  systems  greatly  benefited  from 
interaction  with  instructors  and  other  students.  This  finding  creates  a  challenge  for 
"instructorless"  training  for  soldiers  who  do  not  have  a  base  level  of  skill  or  knowledge 
in  the  subject  area.  IMI  was  generally  found  to  be  more  effective  for  incremental 
learning,  as  opposed  to  initial  learning,  on  the  automated  systems  the  soldiers  operate 
in  their  military  occupational  specialty  (MOS  92A).  Soldiers  with  some  job  experience 
faired  better  with  IMI  than  soldiers  with  little  or  no  basic  computer  skills  or  on-the-job 
experience.  Thus,  teaching  initial  skills  to  students  requires  different  types  of 
instruction  than  teaching  students  with  a  foundation  in  the  subject  matter.  Use  of 
multimedia  instruction  for  initial  skills  training  may  not  be  worth  the  cost,  since  it  has 
to  be  constructed  to  provide  the  structure,  motivation,  and  task  focusing  that  the 
instructor  would  provide  in  a  more  traditional  classroom  setting. 

For  computer-based  or  assisted  learning,  including  online  instruction,  students 
require  basic  computer  skills.  The  Army  can  reduce  attrition  and  increase  soldier 
training  success  by  leveraging  knowledge  on  computer  backgrounds  and  basic  computer 
literacy.  By  identifying  soldiers  who  enter  the  service  with  minimal  computer 
experience,  the  required  basic  skills  can  be  provided  to  them  before  the  main  course  of 
training,  thereby  reducing  recycles  and  failure  rates. 

Summary  of  Recent  Trends  in  the  DL  Literature 

The  pace  of  distance  education  is  increasing.  There  are  increasing  numbers  of  new 
DL  programs  both  nationally  and  internationally.  Technology  is  moving  faster  than  the 
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pedagogy.  Research  on  how  to  use  this  technology  in  teaching  is  lagging  far  behind  the 
pace  of  the  development  of  the  technology.  Good  teaching  practices  have  taken  decades 
to  develop.  Accelerating  this  process  to  keep  pace  with  technology  is  a  big  challenge 
(Faculty,  1999). 

Technology  development  will  continue  to  move  quickly  (Kurzweil,  1999).  Things  like 
streaming  videos  will  be  very  common  in  instructional  material  in  the  near  future,  and, 
as  the  technology  gets  better,  the  costs  of  that  technology  will  be  decreasing.  Many  of 
the  different  technologies  being  explored  today  will  change  dramatically,  or  go  away, 
and  still  unforeseen  technologies  will  emerge  tomorrow  to  complicate  plotting  a  course 
for  today.  Whatever  the  Army  does  with  DL,  it  should  be  positioned  for  constant  change 
and  upgrading. 

Recent  literature  cites  a  need  for  tighter  research  controls  to  better  quantify 
differences  between  distance  learning  and  traditional  training  program  outcomes.  Of 
all  the  different  trials,  experiments,  and  technology  insertion  efforts  going  on,  the 
quality  of  the  planning,  equipment,  teaching  skill,  and  other  factors  vary  widely. 
Comparisons  have  to  be  made  carefully  and  conclusions  made  cautiously  because  of  the 
wide  variance  in  quality  and  circumstances  pertaining  to  DL  programs  (Wisher  et  al., 
1999). 

We  are  witnessing  an  evolution  in  the  definition  of  what  constitutes  a  learning  event 
(Raschke,  1999).  As  the  learning  event  is  being  redefined,  skills  required  for  learning 
are  being  delineated:  organizing  information,  inquiry  skills,  research  skills,  information 
literacy,  independent  study,  communication  skills,  and  team  problem-solving  skills. 

Development  of  new  benchmarks  for  assessment  and  evaluation  of  DL  programs 
have  been  pursued  in  the  last  couple  of  years  (Policy,  2000).  Both  students  and 
instructors  have  to  learn  new  ways  to  interact  and  to  learn  and  teach.  Additionally, 
instructors,  students,  and  administrators  all  see  different  things  in  the  same  events. 
External  evaluators,  when  used,  also  often  see  things  differently  from  all  trial 
participants.  Most  carefully  planned  pilot  projects  have  high  quality,  motivated 
instructors,  as  well  as  technical  support.  Once  those  move  to  everyday  practice,  those 
factors  that  may  have  greatly  influenced  the  outcome  of  the  trials  will  be  replaced  with 
typical  teachers  and  support.  The  effect  of  that  shift  is  largely  unknown,  but  could  be 
substantial  (Hara  and  Kling,  2000). 

A  growing  conservatism  with  regard  to  cost  benefits  is  evident  in  recent  literature 
(Faculty,  1999).  Cost  savings  are  not  clear  for  several  reasons:  explosive  technology 
growth,  coupled  with  competitive  market  factors;  the  emergence  of  a  global 
economy/marketplace;  and  although  numerous  trials  and  experiments  are  underway,  it 
is  still  too  early  to  know  what  will  work  and  what  will  not.  It  is  known  that  a 
significant  up-front  investment  is  required  for  DL,  both  in  technology  and  in 
instructional  design  and  instructor  preparation.  At  present,  most  savings  projections 
are  in  the  delivery  of  DL,  not  the  development,  although  this  may  change  with  evolving 
technologies. 
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Equity  of  access  issues  are  of  growing  concern  and  are  being  studied,  both  nationally 
and  internationally  (Gladieux  and  Swail,  1999). 

The  trend  toward  a  global  economy/society  is  emerging  (Council,  2000). 

The  effectiveness  of  DL  is  dependent  on  the  quality  of  the  course  materials,  the 
skills  of  the  instructor  to  create  effective  learning  situations,  learning  skills  of  the 
student,  and  the  quality  of  the  equipment,  technical  support,  and  courseware  (Policy, 
2000).  Not  all  DL  courses  require  the  highest  technological  applications  to  be  successful 
(Wisher  and  Priest,  1998). 

Implications  for  Army  Training 

As  a  final  summary,  some  implications  for  Army  training  from  this  recent  review  of 
DL  literature  are  provided. 

Perhaps  the  most  striking  phenomenon  evident  in  the  literature  is  the  rapid 
advance  in  technology.  Clearly,  policy-makers,  program  designers,  and  training 
developers  are  all  trying  to  keep  abreast  of  a  fast-moving  train.  Changes  are  happening 
much  faster  than  in  even  the  recent  past.  The  Army's  (and  society's  in  general)  need  to 
invest  in  new  innovation  requires  significant  monetary  investments.  While  the  Army 
must  invest  in  technology  in  order  to  advance  its  skills  and  take  advantage  of  new 
innovation,  the  need  is  balanced  by  the  potential  risk  of  over-investing  in  a  technology 
that  soon  becomes  outdated  or  ineffective.  Information  technology  CEOs  from  Silicon 
Valley  suggested  that  the  military  develop  several  prototype  systems  for  fielding  and 
not  try  to  go  with  a  "one  size  fits  all"  approach  for  technology  investment  (West,  1999). 

Second,  the  literature  is  in  general  agreement  that  training  effectiveness  of  DL 
courses  can  be  at  least  equivalent  to  that  of  traditional  classroom  instruction.  One  of 
the  primary  benefits  of  DL  with  implications  for  Army  training  may  well  be  the 
increased  readiness  of  our  troops  due  to  the  advantages  of  being  able  to  train  troops 
almost  anywhere  in  the  world,  at  anytime,  and  just-in-time  (Metzko  et  al.,  1996).  Many 
of  the  battlefield  automated  systems  are  very  complex.  Studies  have  documented  the 
highly  perishable  nature  of  complex  automation  skills  (Throne  and  Lickteig,  1997). 

With  DL,  skills  can  be  trained,  retrained,  and  refreshed  with  expediency.  This  could 
give  the  US  the  advantages  it  needs  in  the  coming  decades  to  combat  terrorism  and  to 
deploy  rapidly  and  efficiently  to  global  hotspots. 

The  tremendous  advantages  available  from  technology  are  lost  if  the  learning  tools 
that  truly  take  advantage  of  the  new  media  are  not  being  incorporated.  It  is  not  enough 
to  simply  transfer  programs  of  instruction  designed  for  traditional  classroom  delivery 
onto  the  Web.  Courses  need  to  be  redesigned  to  take  into  account  the  inherent  design 
advantages  (or  disadvantages)  of  the  new  technology  over  the  old  technology  (Bates, 
1995).  The  literature  also  points  out  that  the  requirements  for  initial  skills  training 
vary  from  refresher  training  requirements  (George  et  al.,  2001;  Hara  and  Kling,  2000; 
Lee  et  al.,  1996).  Multimedia  training  programs  may  need  to  provide  more  aid  to 
students  with  less  initial  knowledge,  including  more  goal-directedness,  feedback  on 
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activities,  and  connections  with  previous  knowledge.  The  Army  recruits  soldiers  from  a 
diverse  and  heterogeneous  population  with  varying  levels  of  computer  experience 
(Bachman  et  al.,  2000).  Soldier  training  success  can  be  increased  by  identifying  soldiers 
with  little  or  no  computer  experience  and  providing  them  with  computer  literacy 
training  to  get  them  off  to  a  better  start. 

Computer  literacy  is  now  considered  an  essential  battlefield  skill.  The  Silicon  Valley 
CEOs  mentioned  earlier  in  this  report  recommended  that  IT  investment  at  the  tactical 
fighting  level,  what  they  referred  to  as  a  "core  competence,"  be  reappraised,  given  the 
huge  disparity  between  the  amount  of  IT  investment  for  garrison  (41  percent  of  total  IT 
budget)  and  the  tactical  level  (17  percent)  (West,  1999). 

History  suggests  that  the  introduction  of  new  technologies,  such  as  the  new  digital 
information  technologies,  is  usually  accompanied  by  major  changes  in  the  organization 
of  work  (Bates,  2000).  The  universities  are  currently  undergoing  substantial 
organizational  changes  to  accommodate  the  emerging  DL  infrastructures,  including 
providing  for  a  responsive  online  service  environment  for  such  student  services  as 
admissions,  advising,  library  services,  disability  services,  tutorial  services,  technical 
support,  and  certification.  This  is  a  challenge  the  military  also  faces.  A  totally  online 
environment  requires  an  extensive  technical  and  support  infrastructure,  to  include  new 
strategic  partnerships  that  allow  for  these  services. 
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be  leveraged  to  design  training  in  such  a  way  as  to  increase  soldier 
training  success  and  to  reduce  attrition  rates. 

2.  Bachman,  J.  G.,  Segal,  D.  R.,  Freedman-Doan,  P.,  and  O'Malley,  P.  M.  (2000), 
Who  Chooses  Military  Service?  Correlates  of  Propensity  and  Enlistment  in 
the  US  Armed  Forces,  Military  Psychology,  12(1),  1-30. 

This  article  examines  factors  correlated  with  plans  for  military  service  and  actual 
enlistment.  It  reports  bivariate  and  multivariate  regression  analyses  using  survey  data 
from  nationwide  samples  totaling  more  than  100,000  US  high  school  seniors  (classes  of 
1984-1991),  and  follow-up  data  from  a  subsample  of  more  than  15,000  of  the  high  school 
seniors  obtained  1  to  2  years  after  graduation. 

Three  broad  categories  of  potential  correlates  of  military  propensity  among  high 
school  seniors  were  defined:  (a)  family  and  demographic  background,  (b)  educational 
background  and  aspirations,  and  (c)  a  wide  range  of  values,  attitudes,  and  behaviors. 

All  analyses  were  conducted  separately  for  men  and  women  due  to  substantial 
differences  in  enlistment  levels  and  in  a  variety  of  other  gender  differences  in  correlates 
of  enlistment. 

Racial  Differences:  African  American  students  showed  by  far  the  highest  average 
levels  of  propensity  and  enlistment;  Hispanic  students  were  next  highest,  and  White 
students  were  lowest. 

Parents'  Educational  Level:  Higher  levels  of  parental  education  are  associated  with 
lower  levels  of  military  propensity  and,  as  parental  education  rises,  the  likelihood  of 
enlistment  grows  progressively  smaller. 

Plans  for  College:  Those  students  expecting  definitely  to  complete  college  show  the 
lowest  rates  of  propensity  and  enlistment,  and  those  expecting  not  to  get  a  college 
degree  show  the  highest  rates.  Among  women,  the  relations  between  college  plans, 
propensity,  and  enlistment  are  weak  and  nonlinear. 
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The  lower  an  individual's  high  school  grade  average,  the  more  likely  he  or  she  is 
inclined  toward  military  service. 

Attitudes,  Values,  and  Behaviors:  Attitudes  about  military  service  and  performance 
were  found  to  have  a  linear  correlation  with  propensity  and  enlistment  -  the  more  pro¬ 
military  the  attitude,  the  higher  the  levels  of  propensity  and  rates  of  enlistment. 
Behaviors  found  relevant  to  propensity  included  frequency  of  vigorous  exercise  (relevant 
to  meeting  military  fitness  requirements)  and  substance  use/abuse.  All  of  the  substance 
use  measures  show  extremely  low  relations  with  military  propensity  and  enlistment. 

The  authors  note  that  the  factors  that  had  influenced  which  men  served  in  the 
military  during  the  early  years  of  the  volunteer  force  have  persisted  into  the  second 
decade  of  the  volunteer  force.  Men  who  serve  come  disproportionately  from  minority 
racial  and  ethnic  groups,  below-average  socioeconomic  background,  nonsuburban 
residence,  and  regions  other  than  the  Northeast  or  West.  They  tend  to  have  received 
mediocre  grades  in  high  school,  to  come  from  nonacademic  programs,  and  to  have  low 
college  aspirations.  Women  have  a  much  lower  likelihood  of  enlisting  and  although 
propensity  to  enlist  is  the  strongest  predictor  of  their  service,  it  is  a  weaker  predictor  for 
women  than  it  is  for  men. 

Keywords:  Course  Design,  Specialized  Literature 

As  the  battlefield  goes  digital,  what  is  the  effect  on  the  soldier,  upon  the 
leaders,  and  upon  the  mechanisms  of  warfare  itself?  The  following  book  of 
essays  written  by  military  specialists  and  officers  explores  these  and  other 
issues  related  to  digitization. 

3.  Bateman,  R.  L.  I.  (Ed.)  (1998),  Digital  War:  A  View  from  the  Front  Lines , 
Novato,  CA,  Presidio  Press. 

Digitization,  the  latest  revolution  in  the  course  of  military  affairs,  is  predicted  to 
change  almost  every  aspect  of  warfare.  This  book  contains  a  series  of  reports  written  by 
military  specialists  and  officers  on  the  impacts  of  digitization  on  warfare,  the  battlefield, 
and  the  soldier.  Bateman  poses  the  question  in  the  first  chapter,  entitled  Pandora’s 
Box,  of  what  impact  digitization  will  have  on  two  constants  of  military  affairs:  fear  and 
leadership.  What  is  the  impact  of  the  issue  of  information  processing  upon  leaders  and 
leader  development  due  to  the  phenomenal  explosion  of  information  now  made  available 
through  digitization?  In  other  essays,  officers  examine  the  abilities  that  computers  and 
digital  communications  provide  the  soldier  and  how  they  might  be  employed  to  create 
greater  velocity  in  warfare  (Robert  Leonhard),  examine  the  potential  of  digitization  and 
simulation  for  training  the  National  Guard  (Mike  Pryor),  and  examine  the  costs  for  new 
technologies  and  probable  future  equipment  scenarios  given  current  budgetary  realities 
(Daniel  Bolger).  This  book  provides  a  multifaceted  military  perspective  on  the  future 
digital  battlefield. 

Keywords:  Automation  Skills,  Specialized  Literature 


20 


One  of  the  pioneers  of  DL,  Tony  Bates  provides  educators  and  decision¬ 
makers  a  framework  in  which  to  create ,  organize ,  and  assess  a  DL 
program  in  the  following  two  selections  from  the  literature. 

4.  Bates,  A.  W.  T.  (1995),  Technology ,  Open  Learning ,  and  Distance  Education , 
London  and  New  York,  Routledge. 

This  book  provides  a  methodology  for  decision-makers  by  which  they  can  arrive  at 
their  own  conclusions  regarding  DL  technology  selection.  The  author  provides  a 
decision-making  framework  based  on  a  series  of  questions  that  each  institution  needs  to 
ask,  grouped  under  the  following  criteria:  access,  costs,  teaching  and  learning, 
interactivity  and  user-friendliness,  organizational  issues,  novelty,  and  speed 
(ACTIONS). 

Keywords:  Costs,  Course  Design,  Specialized  Literature 

5.  Bates,  A.  W.  T.  (2000),  Managing  Technological  Change:  Strategies  for 
College  and  University  Leaders,  San  Francisco,  Jossey-Bass  Publishers. 

A.  W.  Tony  Bates  has  over  30  years  experience  as  an  educator  in  the  field  of  DL  and 
IT.  He  has  worked  with  the  British  Open  University,  with  the  Open  Learning  Agency 
in  British  Columbia,  and  currently  with  the  University  of  British  Columbia,  where  he  is 
responsible  for  developing  distance  education  programs. 

Bates  summarizes  his  findings  in  new  technology  for  education  as  follows:  (1)  New 
technologies  widen  access  to  new  learners,  increases  flexibility  for  traditional  students, 
and  improves  the  quality  of  teaching  by  achieving  higher  levels  of  learning,  such  as 
analysis,  synthesis,  problem  solving,  and  decision  making.  (2)  New  technologies  are 
unlikely  to  lead  to  a  reduction  in  spending  by  higher  education  institutions,  at  least  in 
the  short  run,  because  of  the  high  and  recurrent  cost  of  investment.  However,  improved 
cost  effectiveness  can  be  realized  by  enabling  new  target  groups  to  be  reached  and 
higher  quality  learning  outcomes  to  be  gained  at  a  lower  marginal  cost  per  student  than 
through  conventional  classroom  methods.  To  achieve  gains  in  cost  effectiveness, 
teaching  and  learning  need  to  be  substantially  reorganized.  (3)  History  suggests  that 
the  introduction  of  new  technology  is  usually  accompanied  by  major  changes  in  the 
organization  of  work. 

Keywords:  Benefits,  Costs,  Course  Design,  Specialized  Literature 

6.  Bishop,  T.  and  SchWeber,  C.  (2000,  September  17-20,  2000),  UMUCfs  Online 
MBA  Program:  A  Case  Study  of  Cost-Effectiveness  and  the  Implications  for 
Large-Scale  Programs .  Paper  presented  at  the  Asynchronous  Learning 
Networks  (ALN)  Workshop  2000:  Learning  Effectiveness,  Faculty  Satisfaction, 
and  Cost  Effectiveness,  Lake  George,  NY. 

This  report  describes  the  cost  analysis  case  study  of  the  online  Master  of  Business 
Administration  (MBA)  program  at  University  of  Maryland  University  College  (UMUC). 
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In  1999  to  2000,  online  enrollments  at  UMUC  had  grown  to  40,000  worldwide,  or  40 
percent  of  total  enrollments,  and  there  were  more  than  1,200  different  course  offerings 
at  the  undergraduate  and  graduate  levels.  Large-scale  ALN  operations  require  broad 
institutional  support  -  such  as  the  IT  infrastructure,  student  services,  library  and 
information  services  -  which  has  important  cost  implications. 

The  graduate  school  developed  a  formal  program  proposal  that  included  a  cost 
analysis  to  project  the  fiscal  viability  of  the  program.  The  analysis  of  cost  recovery  over 
a  5-year  period  projected  a  net  loss  in  Year  1  with  modest  net  profit  in  Year  2, 
continuing  over  the  next  3  years.  The  cost  analysis  estimated  course  development  and 
delivery  costs  based  on  varying  class  enrollments  of  15,  20,  25,  and  30  students.  Fixed 
costs  included  faculty  salary,  faculty  training,  course  development,  instructional 
designer,  course  evaluation  (a  mere  $200),  peer  review,  program  coordinator,  Web 
administration,  and  marketing  costs.  Variable  costs,  which  fluctuate  depending  on  the 
number  of  students,  included  electronic  library  costs,  online  student  services,  IT  help 
desk  support,  and  administrative  overhead.  The  preliminary  cost  analysis  indicated 
that  a  small  change  in  class  size  could  impact  the  program  profit  margin. 

The  online  MBA  program  was  initially  offered  in  1999  to  2000.  The  preliminary 
data  indicate  that  actual  enrollments  were  92  percent  higher  than  projected,  and  as 
such,  revenue  exceeded  the  projection  prepared  in  the  original  cost  analysis  and  the 
program  experienced  a  positive  return  on  investment  in  the  first  year.  The  enrollment 
growth  requires  additional  resources,  including  full-time  faculty  and  staff,  and  teaching 
assistants  are  being  added  in  Year  2  to  help  faculty  with  the  additional  workload 
demands. 

Keywords:  Costs,  Specialized  Literature 

The  number  of  accredited  DL  colleges  is  over  900  and,  the  number  of 
accredited,  degree  and  certificate  DL  programs  has  reached  over  1,200  u i 
the  US  alone.  With  all  of  this  activity ,  a  whole  new  industry  has  sprung 
up  in  the  development  and  marketing  of  DL  courseware . 

7.  Blumenstyk,  G.  (1999),  The  Marketing  Intensifies  in  Distance  Learning, 
Chronicle  of  Higher  Education,  9  April  1999. 

According  to  this  article,  the  marketing  brochures,  information  kits,  and  compact 
disk-read  only  memory  (CD-ROMs)  that  distance  education  companies  use  to  promote 
course  design  tools  and  other  services  are  going  to  be  flying  fast  and  furious  over  the 
coming  months.  Such  companies  as  Blackboard,  Convene,  Embanet,  and  Real 
Education  are  maneuvering  on  multiple  fronts  to  gain  more  visibility  for  their  products. 
Real  Education,  of  Denver,  is  quadrupling  its  sales  force.  Blackboard,  in  Washington,  is 
wooing  professors  with  free  Web  sites  for  online  courses.  Convene,  in  San  Francisco, 
has  just  hired  a  top  official  away  from  the  University  of  Phoenix's  6,000-student 
distance  education  operation  to  head  up  marketing.  The  Connecticut  State  University 
System  recently  signed  a  3-year  contract  with  Real  Education  after  its  Trustees  decided 
that  the  State  should  use  technology  to  make  education  more  widely  available 
throughout  the  state. 
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The  success  of  Britains'  Open  University  has  been  attributed  in  part  to 
the  emphasis  on  meeting  the  needs  of  nontraditional  students ,  utilizing  a 
mix  of  flexible  thinking  and  creativity  to  make  higher  education  accessible 
to  new  audiences . 

8.  Blumenstyk,  G.  (1999),  Distance  Learning  at  the  Open  University,  Chronicle 
of  Higher  Education,  23  July  1999. 

Since  1971,  more  than  2  million  students  have  taken  a  course  using  the  Open 
University's  virtual  education  model  and  more  than  227,000  people  have  earned  their 
degrees  at  Britains'  once  controversial  Open  University.  The  school  uses  old-fashioned 
correspondence  course  methods  combined  with  multimedia  materials,  personal  tutoring, 
and  in  some  instances,  a  weekend  or  weeklong  class  session  at  a  residential  school. 

The  Open  University  challenged  social  perceptions  of  what  higher  education  was  for, 
designing  its  courses  for  part-time  students  and  working  adults  rather  than  the 
traditional  high-achieving  18-year-olds.  With  its  use  of  distance  education  and  flexible 
standards,  the  Open  University  made  higher  education  much  more  available  to  people. 
Nontraditional  part-time  students  remain  the  core  constituency:  the  median  age  of  its 
entering  undergraduates  is  34  and  a  third  of  its  students  already  have  a  degree. 

Created  in  the  1960s,  the  Open  University  is  a  proven  success  story  that  employs  900 
full-time  faculty  members  and  about  7,000  tutors  known  as  "associate  lecturers,"  and 
has  an  annual  budget  of  215  million  pounds. 

The  design  of  an  Open  University  course  is  a  sophisticated,  resource  intensive 
collaboration,  usually  taking  2  to  3  years  to  develop  and  they  are  very  expensive  to 
produce.  The  materials  for  one  course  "Pacific  Studies,"  included  four  books,  several 
study  guides,  five  specially  produced  half-hour  television  (TV)  shows,  and  a  set  of 
audiotapes.  A  40-person  "course  team"  oversaw  the  development  of  "Pacific  Studies." 
The  Open  University  takes  pride  in  the  quality  of  the  TV  shows,  shooting  scenes  from 
three  continents  for  Pacific  Studies.  Generally,  the  programs  are  aired  on  the  British 
Broadcasting  Company  (BBC)  during  off  hours,  but  sometimes  the  courses  make  it  into 
prime  time.  The  core  of  the  classes  is  the  written  material,  and  the  books,  often  used  at 
other  universities  and  purchased  by  some  45,000  readers  annually  who  are  not  formally 
enrolled  in  an  Open  University  course,  are  published  with  collaboration  with  outside 
publishers.  The  Open  University  owns  and  keeps  all  the  intellectual-property  rights 
from  its  course  materials.  Courses  are  generally  designed  for  an  8-year  life  span,  with 
major  revisions  planned  at  3-year  intervals.  The  course  team  also  meets  regularly  to 
create  new  examinations  and  assignments. 

Tutors  interface  between  the  university  and  the  students.  They  are  the  students' 
translators  and  guides,  helping  to  explain  material  and  answer  questions  through 
regularly  scheduled  tutorial  sessions,  by  telephone,  and  by  e-mail.  The  tutors  grade  the 
papers  devised  and  prescribed  by  the  course  teams,  referred  to  as  "tutor-marked 
assignments."  Tutors,  many  of  them  professionals  in  business  or  social  work  or 
moonlighting  professors  from  other  universities,  are  hired  under  8-year  contracts. 
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9.  Brown,  G.  and  Mack,  M.  (1999),  The  Difference  Frenzy  and  Matching 
Buckshot  with  Buckshot,  The  Technology  Source ,  5. 

The  recent  report  entitled  ’’What’s  the  Difference"  by  Phipps  and  Merisotis  on  the 
effectiveness  of  DL  in  higher  education  comes  under  fire  in  this  article.  The  authors  fail 
to  "unpack"  the  real  questions  embedded  in  their  assertion  that  an  entire  body  of 
research  needs  to  be  developed  to  determine  if  students  that  participate  for  their  whole 
program  in  DL  compare  favorably  with  students  taught  in  the  conventional  classroom, 
thus  making  their  premise  for  evaluating  the  research  on  distance  education  seem 
overly  simplistic.  Phipps  and  Merisotis  lament  the  paucity  of  "true,  original  research..." 
without  defining  what  they  mean  by  true  or  original.  Brown  and  Mack  point  out  that 
most  of  the  criticisms  in  the  report  could  be  and  are  applicable  to  conventional  face-to- 
face  models  of  educational  practice  and  that  clear,  comprehensive  evidence  is  rarely 
obtainable  in  the  "complex,  messy  world  of  teaching  and  learning,  even  after  decades  of 
educational  research." 

Keywords:  Benefits,  Specialized  Literature 

A  software-based  cognitive  tutor  developed  by  Carnegie  Mellon 
researchers  has  been  shown  to  improve  standard,  math  test  scores  and 
significantly  increase  student  achievement  in  algebra  and  geometry  over 
traditionally  delivered  classes. 

10.  Carnegie  Mellon  (1999),  Carnegie  Mellon  Forms  Carnegie  Learning  Inc . 
[Carnegie  Mellon  Web  site,  Computers  &  Information  Technology,  What’s  New]. 
Available:  http://whatsnew.andrew.cmu.edu/Computerslnformation.qrv. 

Carnegie  Mellon  has  formed  Carnegie  Learning,  Inc.,  a  new  company  to  market  its 
series  of  software-based  cognitive  tutors  that  have  been  teaching  algebra  and  geometry 
to  middle  through  college  students  since  1993.  The  tutors  are  backed  by  nearly  2 
decades  of  research  and  more  than  1 1  million  in  funding  from  government  and 
foundations.  The  tutors  have  been  used  by  thousands  of  students  and  have  been  shown 
to  improve  standard  test  scores  and  significantly  increase  student  achievement  gains 
over  traditionally  delivered  classes.  Experts  say  the  cognitive  tutors  are  one  of  the  few 
educational  products  available  today  that  have  the  research  behind  them  to  prove  they 
work.  The  original  tutor  was  developed  by  John  R.  Anderson,  a  professor  of  psychology 
and  computer  science  at  Carnegie  Mellon.  It  is  built  around  the  theory  that  students 
learn  best  by  actively  participating  in  the  learning  process  rather  than  receiving  static, 
verbalized  information.  Tutor-taught  students  learn  material  in  one-third  less  time 
than  in  traditional  learning  situations.  They  score  15  percent  higher  on  standardized 
tests  and  100  percent  better  than  their  peers  in  applying  mathematics  in  authentic 
problem  solving  situations.  Robert  A.  Longo,  a  marketing  executive  and  former 
educator  has  been  named  president  and  chief  executive  officer  of  the  new  firm. 
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11.  Carnevale,  D.  (2001),  Assessment  Takes  Center  Stage  in  Online  Learning, 
Chronicle  of  Higher  Education,  13  April  2001,  A43-A45. 

Assessment  is  becoming  an  important  aspect  of  a  distance  education  program,  as 
online  educators  seek  to  prove  that  they  are  teaching  effectively.  Two  distance 
education  institutions  that  are  creating  assessment  techniques  are  the  Western 
Governor's  University  and  the  University  of  Phoenix.  According  to  Peter  Ewell,  senior 
associate  at  the  National  Center  for  Higher  Education  Management  Systems,  all  of 
higher  education  is  moving  toward  outcome-based  assessments,  a  movement  led  by 
online  education.  Critics  often  hold  distance  education  to  a  higher  standard  than 
traditional  education  when  judging  quality,  probably  because  of  its  relative  newness. 
There  are  also  pressures  being  placed  on  traditional  education  to  do  more  to  assess 
quality,  from  accreditation  agencies,  state  governments,  and  policy  boards.  Both 
traditional  and  distance  education  institutions  are  struggling  to  figure  out  how  to  use 
the  results  of  assessment  examinations. 

Keywords:  Benefits,  Practitioner  Literature 

12.  Carr,  S.  (2000),  US  Army  Chooses  PricewaterhouseCoopers  to  Run  Big 
Distance-Education  Project,  Chronicle  of  Higher  Education  (Daily  News 
12/15/00),  5. 

The  consulting  company,  PricewaterhouseCoopers,  has  been  selected  to  be  the 
integrator/coordinator  for  the  $453  million  project  to  deliver  DL  courses  to  soldiers  all 
over  the  world.  The  company  has  formed  a  team  of  10  companies  and  29  colleges  to 
work  together  on  the  initial  offering. 

The  project,  called  Army  University  Access  Online,  will  provide  DL  courses  to 
soldiers  at  little  or  no  charge.  The  first  courses  will  be  offered  at  three  Army  bases  next 
month,  with  an  estimated  15,000  students.  The  initial  29  academic  institutions  were 
selected  primarily  for  their  experience  in  working  with  the  military,  as  well  as  providing 
a  diverse  mixture  of  institutions.  The  company  will  begin  expanding  the  pool  of 
academic  institutions  in  subsequent  offerings.  This  article  includes  a  list  of  the  29 
academic  institutions  and  the  10  corporate  partners,  which  include  Blackboard, 
Peoplesoft,  and  Smartthinking. 

Keywords:  Benefits,  Practitioner  Literature 

13.  Carr,  S.  (2001),  Is  Anyone  Making  Money  on  Distance  Education?, 

Chronicle  of  Higher  Education,  16  February  2001,  10. 

Six  new  studies  commissioned  by  the  Alfred  P.  Sloan  Foundation  analyzed  the  costs 
of  online  education  and  its  potential  profitability.  The  six  universities  that  participated 
in  the  cost  studies  all  had  received  grants  from  the  Foundations'  ALN  to  develop  online 
programs.  Most  of  the  reports  indicate  that  the  universities  are  near  the  break-even 
point  with  their  DL  programs.  The  studies  also  found  that  defining  costs  is  a  difficult 
process,  that  university  accounting  procedures  were  not  always  well-suited  to  a  case- 
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study  approach,  that  small  variations  in  class  size  could  affect  profitability,  and  that 
costs  are  not  likely  to  be  significantly  cheaper  than  traditional  delivery.  While  costs  of 
technology  and  technical  support  required  for  DL  programs  are  greater  than  traditional 
programs,  the  physical  infrastructure  that  is  needed  to  support  traditional  programs 
will  ultimately  make  the  difference  in  profitability.  Unlike  technology,  the  cost  of  land, 
buildings,  and  utilities  do  not  go  down  over  time. 

Keywords:  Costs,  Practitioner  Literature 

14.  Chickering,  A.  W.  and  Ehrmann,  S.  C.  (1997),  Implementing  the  Seven 
Principles:  Technology  as  Lever,  American  Association  for  Higher  Education, 

6  pp. 

In  1987,  the  American  Association  for  Higher  Education  (AAHE)  Bulletin  published 
"Seven  Principles  for  Good  Practice  in  Undergraduate  Education."  This  article  updates 
the  seven  principles  with  respect  to  technology  and  the  new  learning  tools. 

The  seven  principles  are: 

1.  Good  Practice  Encourages  Contacts  Between  Students  and  Faculty.  The 
update  concludes  that  new  communication  technologies  can  increase  access  to  faculty 
members  and  provide  for  joint  problem  solving  and  shared  learning  and  can  strengthen 
faculty  interactions  with  all  students,  but  especially  with  shy  students  or  non-English 
native  speakers. 

2.  Good  Practice  Develops  Reciprocity  and  Cooperation  Among  Students. 
Learning  is  enhanced  when  it  is  more  of  a  team  effort  than  a  solo  race.  The  update 
notes  that  the  increased  opportunities  for  interaction  noted  above  with  faculty  apply 
equally  to  communication  with  fellow  students.  Study  groups,  group  problem  solving, 
collaborative  learning,  and  feedback  can  all  be  strengthened  through  communication 
tools  such  as  e-mail. 

3.  Good  Practice  Uses  Active  Learning  Techniques.  Learning  by  doing,  or 
apprentice-like  learning,  has  been  supported  by  many  traditional  technologies,  research 
libraries,  laboratories,  art  and  architectural  studios,  athletic  fields.  Newer  technologies 
can  also  enrich  and  expand  these  opportunities.  Time-delayed  exchange  and  real-time 
conversation  technologies  can  also  help  students  to  talk  about  what  they  are  learning, 
write  reflectively  about  it,  relate  it  to  past  experiences,  and  apply  it  to  their  daily  lives. 

4.  Good  Practice  Gives  Prompt  Feedback.  In  getting  started,  students  need  help 
in  assessing  their  existing  knowledge  and  competence,  and  in  classes,  students  need 
frequent  opportunities  to  perform  and  to  receive  feedback  on  their  performance.  New 
technologies  can  provide  feedback  in  many  ways,  some  obvious,  and  some  more  subtle. 
Computers  can  provide  rich  storage  and  easy  access  to  student  products  and 
performances,  can  keep  track  of  early  efforts  and  thereby  provide  a  benchmark  to 
demonstrate  gains  in  knowledge,  competence,  or  other  valued  outcomes. 
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5.  Good  Practice  Emphasizes  Time  on  Task.  New  technologies  can  dramatically 
improve  time  on  task  for  students  and  faculty  members.  DL  can  also  save  hours 
otherwise  spent  commuting  to  and  from  campus,  finding  parking  spaces,  going  to  the 
library  to  get  access  to  important  resources,  etc. 

6.  Good  Practice  Communicates  High  Expectations.  Expecting  students  to 
perform  well  becomes  a  self-fulfilling  prophecy.  Students  may  be  motivated,  for 
example,  by  knowing  that  their  work  may  be  published  on  the  Web.  General  criteria 
can  be  illustrated  with  online  samples  of  excellent,  average,  and  faulty  performance. 

7.  Good  Practice  Respects  Diverse  Talents  and  Ways  of  Learning.  Technologies 
can  provide  different  learning  repertoires  for  different  learning  styles  and  learning 
curves. 

Keywords:  Benefits,  Course  Design,  Instructor,  Specialized  Literature 

Industry  experts  in  the  field  of  automation  training  point  out  the 
importance  of  training  for  the  success  of  a  new  automation  project,  and 
indicate  that  at  least  6-10  percent  of  the  new  projects'  budget  must  be 
allocated  to  training  for  the  effort  to  be  successful. 

15.  Compton,  J.  (1999),  CRM  Training  by  the  Book,  Sales  and  Field  Force 
Automation  -  The  Executive's  Guide  to  Customer  Relationship  Management. 

This  article  highlights  the  importance  of  training  for  the  success  of  a  new  sales 
automation  project.  Training  is  a  prerequisite  for  success;  without  it  your  project  can 
potentially  cost  more  in  the  long  run  and  foster  discontent  that  can  jeopardize  the  unity 
of  your  sales  organization.  Its  hard  to  measure  what  you  are  getting  for  training 
dollars,  and  top  management  is  often  reticent  to  put  a  lot  of  money  in  when  they  cannot 
see  what  they  are  getting,  says  Thomas  Minero,  president  of  Training  Resources,  Inc.  of 
New  Jersey.  Wendy  Close,  research  director  with  the  Gartner  Group,  recommends 
allocating  6  to  10  percent  of  a  total  sales  and  field  force  automation  (SFA) 
implementation  budget  to  training.  Enterprises  adopting  or  reengineering  their  sales 
automation  solutions  are  starting  to  get  the  message;  a  recent  survey  of  over  200 
companies  by  Insight  Technology  Group  found  that  on  average  5  percent  of  total  SFA 
project  cost  is  going  to  training.  This  new  enthusiasm  for  training  may  be  due  in  part  to 
past  experience;  over  half  of  all  SFA  projects  have  failed  historically.  Further,  firms 
that  develop  and  implement  SFA  projects  are  walking  away  from  projects  in  which  the 
potential  clients  are  unwilling  to  properly  invest  in  training.  Support  costs  are  much 
higher  for  struggling  projects. 

Set  reasonable  goals  for  your  project  and  plan  for  training  at  least  3  to  6  months 
before  delivery,  says  Debra  Exner,  presidents  of  software  training  firm  Exner  and 
Associates.  Strongly  consider  training  management  before  the  pilot  rollout  of  the  new 
system.  How  well  the  manager  is  onboard  supporting  the  new  system  is  highly 
correlated  with  how  fast  the  sales  representatives  pick  it  up.  Finally,  you  will  need  to 
make  an  early  decision  of  how  to  handle  required  basic  computer  skills.  Many  people 
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are  still  clumsy  at  best  with  standard  Windows  interface.  Some  solutions  include 
requiring  students  to  come  to  class  with  a  basic  literacy  of  computer  skills  (which  can  be 
provided  by  paid  classes  at  a  local  community  college  or  training  center)  or  to  extend  the 
training  program  by  a  half-day  and  add  a  required  course  on  general  computing. 

Deciding  whether  to  go  with  your  own  internal  training  staff  or  outsource  the 
training  program  is  a  big  decision  and  will  largely  be  made  by  the  availability  and 
experience  of  your  trainers.  If  you  outsource,  you  have  an  additional  decision  whether 
to  go  with  a  full  service  SFA  consultant  or  a  dedicated  training  firm.  No  matter  who 
does  your  training,  make  sure  they  understand  and  communicate  your  sales  process  as 
part  of  the  education  process.  By  building  training  that  incorporates  real-world  sales 
pi’ocess  examples,  the  chances  are  greatly  improved  that  representatives  will  adopt 
more  of  the  software  in  their  daily  business.  Training  must  answer  "What's  in  it  for 
me?"  and  not  just  provide  a  feature-by-feature  list  of  its  functionality. 

Keywords:  Benefits,  Course  Design,  Practitioner  Literature 

16.  Council,  N.  I.  (2000),  Global  Trends  2015:  A  Dialogue  About  the  Future  with 
Nongovernment  Experts  (NIC  2000-02),  Washington,  DC,  National  Intelligence 
Council,  Central  Intelligence. 

This  report,  produced  by  the  National  Intelligence  Council  in  close  collaboration 
with  a  wide  range  of  experts  outside  of  the  government,  defines  major  drivers  and 
trends  that  will  shape  the  world  of  2015.  Major  drivers  include  demographics,  natural 
resources,  science  and  technology,  the  global  economy,  future  conflict,  and  the  role  of  the 
US. 


Among  the  projections  are  that  the  world  population  in  2015  will  be  7.2  billion,  up 
from  6.1  billion  in  2000,  and  life  spans  are  on  the  increase.  Overall,  food  production  will 
be  adequate,  but  political  instability  and  poor  infrastructure  will  still  result  in  famine  in 
parts  of  Sub-Saharan  Africa.  Energy  supplies  will  be  able  to  meet  the  50  percent 
increase  in  global  energy  demand.  Science  and  technology  will  continue  to  experience 
quantum  leaps  in  development  and  major  paradigm  shifts  will  result.  Older 
technologies  will  continue  lateral  development  into  new  markets,  and  US  adversaries 
around  the  world  will  gain  additional  access  to  early  generation  ballistic  missile  and 
weapons  of  mass  destruction  (WMD)  technologies.  Disaffected  states,  terrorists,  and 
organized  criminals  will  take  advantage  of  the  new  high-speed  information  environment 
and  other  advances  in  technology  to  threaten  stability  and  security  around  the  world. 
Biotechnology  will  drive  medical  breakthroughs. 

The  networked  global  economy  will  be  characterized  by  rapid  and  largely 
unrestricted  flows  of  information,  ideas,  values,  capital,  goods  and  services,  and  people. 
Increasing  reliance  on  computer  networks  is  making  critical  US  infrastructures  more 
attractive  as  targets.  Rapid  advances  in  biotechnology,  nanotechnology,  and  the 
materials  sciences  will  add  to  the  capabilities  of  our  adversaries  to  engage  in  biological 
warfare  or  bioterrorism.  This  report  provides  some  clues  to  the  types  of  global 
encounters  the  Army  of  the  21st  century  will  likely  face. 
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Shirley  M.  Davis,  another  pioneer  in  DL,  discusses  the  new  Age  of 
Learning,  in  which  developments  in  technology  have  brought  dramatic 
changes  in  both  the  learning  needs  and  the  way  learning  opportunities  are 
delivered  in  business,  labor,  government,  and  academia. 

17.  Davis,  D.  S.  M.  (1999),  Ushering  in  the  Learning  Age,  Education  at  a 
Distance,  13(7). 

We  are  becoming  a  society  in  which  continuous  learning  is  an  emerging  mindset  and 
a  key  to  success  in  life  and  career.  Learners  in  the  Age  of  Learning  must  take  increased 
responsibility  for  control  and  direction  of  the  learning  process.  In  addition,  a  new  set  of 
skills  needed  for  the  workforce  of  the  future  have  been  identified,  which  include 
communication  skills,  inquiry  skills,  team  skills,  research  skills,  information  literacy, 
and  problem-solving  skills.  DL  and  developments  in  telecommunications  technologies 
have  not  only  transformed  our  needs  for  education  and  training,  but  have  expanded  our 
capacity  to  respond  to  these  needs.  DL  is  now  a  mainstream  method  of  education  and 
training  that  is  both  flexible  and  responsive  to  learners'  needs.  New  learning  tools  are 
being  developed  that  encourage  collaborative  learning,  that  offer  Web-based 
simulations,  and  that  offer  approaches  that  support  the  new  Age  of  Learning. 

An  example  of  new  course  design  projects  include  those  developed  under  the  NSF- 
funded  SCANS  2000  project  at  Johns  Hopkins.  These  learning  modules  were  developed 
for  use  in  college  classrooms  by  interdisciplinary  teams  of  students  who  address  real 
problems  and  develop  their  collaborative  solution  through  asking  questions  and 
accessing  information  in  a  nonlinear  mode  to  answer  those  questions. 

Keywords:  Course  Design,  Practitioner  Literature 

18.  Davis,  D.  S.  M.  (2000),  Speaking  Personally  -  with  Shirley  M.  Davis,  The 
American  Journal  of  Distance  Education,  14(2),  71-74. 

Shirley  M.  Davis  is  the  director  of  learning  innovations  for  the  PBS  Adult  Learning 
Service  (ALS),  the  nation's  largest  satellite-  and  Web-based  service  for  higher  education. 
In  1999,  Davis  became  president  of  the  United  States  Distance  Learning  Association 
(USDLA). 

In  this  interview  for  the  American  Journal  of  Distance  Education,  Davis  describes 
how  she  is  responsible  for  making  recommendations  for  how  and  when  to  adopt 
emerging  DL  technologies.  One  of  the  missions  of  the  ALS  is  to  help  colleges  use 
technology  to  serve  "nontraditional  students"  and  to  provide  professional  development 
for  faculty  and  administrators.  Another  ALS  initiative  is  to  begin  developing  a 
Web-based  certificate  program  that  prepares  K-12  teachers  to  integrate  technology  into 
their  curricula. 
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PBS,  in  collaboration  with  Intel,  is  becoming  actively  involved  in  "enhanced 
television,"  a  recently  developed  distribution  method  that  enables  supplemental  digital 
information  to  be  broadcast  along  with  a  digital  video  signal  that  can  be  captured  by  a 
computer  for  nonlinear  access.  Over  the  past  5  years,  the  biggest  change  in 
communications  around  the  world  has  been  the  emergence  of  mature  digital 
technologies,  as  nearly  all  communications  move  from  analog  to  digital.  "We  are  just 
beginning  to  understand  their  potential  for  education,"  according  to  Davis. 

Keywords:  Course  Design,  Specialized  Literature 

19.  Dillon,  C.  and  Granger,  D.  (1998),  Guest  Editorial,  The  American  Journal  of 
Distance  Education ,  !i?(l). 

What  is  "the  class"  for?  This  is  the  question  posed  by  this  editorial.  Distance 
education  has  outdistanced  the  distance  and  is  forcing  us  to  question  basic  approaches 
to  teaching  and  learning.  New  technologies  are  making  information  access  and  learning 
possible  in  ways  that  previously  did  not  exist.  Learning  itself  is  becoming  the 
significant  activity,  which  coincides  with  increasing  learning  needs  in  an  information 
society.  The  combination  of  three  factors  -  need,  understanding,  and  capacity  -  has 
resulted  in  a  shift  in  higher  education,  and  a  corresponding  growth  in  virtual 
universities.  "I  am  not  a  teacher  of  things,  I  am  an  orchestrator  of  ideas.  My  university 
is  not  a  physical  plant  with  classrooms  and  trees,  but  a  hub  of  resources  no  longer 
constrained  by  time  and  place.  My  students  are  no  longer  my  students,  but  we  are  all 
students  together."  When  "the  class"  loses  its  meaning,  so  does  the  credit  hour 
production,  the  basic  unit  of  production  in  higher  education.  Can  we  redefine 
productivity  in  education  and  explore  new  ways  of  using  our  resources? 

The  shift  to  alternative  modes  of  teaching  and  learning  is  a  migration,  with 
innovators  and  early  adopters  breaking  ground  for  those  who  advance  more  slowly. 

Keywords:  Course  Design,  Specialized  Literature 

20.  Druckman,  D.  and  Bjork,  R.  A.  (Eds.)  (1994),  Learning ,  Remembering , 
Believing:  Enhancing  Human  Performance ,  Washington,  DC,  National 
Academy  Press. 

This  book  is  the  third  report  of  the  Committee  on  Techniques  for  the  Enhancement 
of  Human  Performance  (CTEHP).  This  14-member  science  committee  was  formed  in 
1985  by  the  National  Research  Council  of  the  National  Academy  of  Sciences  to  evaluate 
a  set  of  specific  "new  age"  techniques  designed  to  enhance  human  performance.  While 
the  members  of  the  committee  change  over  time,  what  has  stayed  largely  in  place  is  its 
method  of  operation.  Site  visits  to  applied  settings,  both  military  and  civilian,  are  an 
intrinsic  aspect  of  the  committee  process,  as  are  briefings  to  the  entire  committee  by 
advocates,  critics,  and  neutral  parties.  Every  member  of  the  committee  must  sign  off  on 
all  of  the  committee's  final  conclusions  and  recommendations;  therefore  highly 
interactive  discussions  are  also  an  indispensable  part  of  the  research  process. 
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This  study  stresses  that  the  purpose  of  training  is  post-training  performance.  The 
authors'  empirically-based  analysis  documents  the  tendency  of  trainees  and  trainers  to 
inadvertently  fool  themselves  and  each  other  about  what  and  how  much  has  been 
learned  and  trained.  The  traditional  climate  of  military  training,  including  pressures 
for  efficiency  and  intolerance  of  errors,  may  reinforce  such  tendencies.  Some  prevailing 
training  conditions  that  enhance  skill  acquisition  may  actually  retard  skill  retention 
and  transfer  to  job  performance.  A  more  effort,  more  effect  approach  to  training  may 
slow  skill  acquisition  but  enhance  retention  and  transfer. 

Keywords:  Benefits,  Course  Design,  Specialized  Literature 

21.  Ewell,  P.  (1999),  Editorial:  No  Significant  Difference?,  Change,  May/June 
1999,  p.  4. 

Keywords:  Benefits,  Specialized  Literature 

This  article  summarizes  the  major  conclusions  of  the  review  of  research  on  DL 
presented  by  Merisotis  and  Phipps  in  their  report  sponsored  by  the  American 
Federation  of  Teachers  and  the  National  Education  Association.  Of  interest  is  the 
editorial  that  introduces  the  piece,  in  which  the  editor  points  out  that  precisely  the  same 
ills  identified  by  Merisotis  and  Phipps  in  the  DL  literature  are  present  in  the  published 
research  about  virtually  every  method  of  instructional  delivery,  including  the  classroom- 
based  ones  we  presently  use.  While  the  call  for  better  research  about  learning  and  its 
causes  are  warranted,  it  applies  to  all  the  ways  teaching  and  learning  are  performed, 
not  just  the  newest.  Second,  the  editorial  points  out  that  the  finding  in  question,  one  of 
"no  significant  difference"  between  DL  and  traditional  classroom-based  learning, 
persists  across  good  studies  and  bad,  for  different  populations  and  different 
technologies.  Wouldn't  a  more  reasonable  predicted  outcome  be  "some  better  and  some 
worse?"  Ewell  points  out  that  it  is  the  robustness  of  these  results  that  is  of  interest. 
What  works  for  one  kind  of  learner  may  not  work  for  another,  a  fact  researched 
thoroughly  about  traditional  classrooms.  Distance  education  is  likely  no  different  and 
further  inquiry  into  the  interactions  among  different  kinds  of  learners  and  specific 
learning  modalities  is  warranted. 

22.  Faculty,  U.  o.  I.  (1999),  Teaching  at  an  Internet  Distance:  The  Pedagogy  of 
Online  Teaching  and  Learning,  the  Report  of  a  1998-1999  University  of  Illinois 
Faculty  Seminar,  University  of  Illinois. 

This  year-long  faculty  seminar  was  convened  at  the  University  of  Illinois  during  the 
1998-1999  academic  year  to  discuss  pros  and  cons  of  implementing  technology  for 
teaching,  particularly  teaching  at  a  distance.  Its  16-member  team  was  evenly  split 
between  "skeptical"  and  "converted"  faculty  and  focused  on  pedagogy,  the  discussions 
centering  on  what  makes  teaching  to  be  good  teaching,  whether  in  the  classroom  or 
online. 

The  seminar  concluded  that  online  teaching  and  learning  can  be  done  with  high 
quality,  particularly  when  professors  make  the  effort  to  create  and  maintain  a  high 
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degree  of  interaction  between  professor  and  student  and  among  the  students 
themselves.  The  seminar  concurred  with  the  "no  significant  difference"  findings 
prevalent  throughout  the  literature;  however,  the  seminar  questioned  the  emphasis 
placed  on  the  method  used  to  deliver  instruction,  noting  that  the  evaluation  of  online 
learning  is  multifaceted  and  subtle,  and  learning  competence  is  only  part  of  the 
evaluation  need.  The  seminar  did  support  the  continued  need  to  examine  learning 
effectiveness  with  at  least  some  of  the  rigor  called  for  by  Phipps  and  Merisotis. 

The  faculty  seminar  identified  a  number  of  policy  issues  for  administrators  that 
included  the  caveat  that  DL  may  not  be  the  moneymaker  anticipated  by  university 
administrators.  The  scenario  once  envisioned  of  hundreds  of  students  enrolling  in  a 
well  developed,  essentially  instructor-free  online  course  does  not  appear  realistic.  With 
some  exceptions,  the  successful  online  course  features  low  student  to  faculty  ratios.  The 
exceptions  involve  extraordinary  amounts  of  the  professor's  time.  Besides  the  initial 
investment  in  technology,  technical  support  for  professors  and  students  and 
maintenance  of  hardware  and  software  are  very  expensive.  Quality  teaching  does  not 
come  cheaply.  Online  instruction  is  not  likely  to  cost  less  than  traditional  instruction. 

Another  important  issue  concerns  the  determination  of  whether  online  teaching  is 
successful.  Before  history  answers  the  question,  the  faculty  members  advocated 
rigorous  comparison  of  learning  competence  between  online  and  traditional  classes, 
pointing  out  that  much  of  the  current  DL  research  is  anecdotal.  High  quality  online 
teaching  is  not  just  a  matter  of  transferring  class  notes  and  lectures  to  the  Internet. 

New  paradigms  of  delivery  and  evaluation  are  needed.  Some  features  to  look  for  in  new 
DL  courses  are  the  strength  of  the  professor-student  and  student-student  interactions, 
the  depth  of  student  engagement  with  the  course  materials,  and  access  to  technical 
support. 

Keywords:  Benefits,  Costs,  Course  Design,  Instructor,  Specialized  Literature 

23.  Geith,  C.  and  Vignare,  K.  (2000,  September  17-20,  2000),  Online  Degree 
Programs:  Service  and  Cost .  Paper  presented  at  the  ALN  Workshop  2000: 
Learning  Effectiveness,  Faculty  Satisfaction,  and  Cost  Effectiveness,  Lake 
George,  NY. 

The  Rochester  Institute  of  Technology  (RIT)  uses  asynchronous  online  learning  to 
support  5,000  students  per  year  in  13  degree  programs  and  19  certificate  programs  (as 
of  31  August  2000).  The  online  programs  are  taught  by  the  same  faculty  as  the  on- 
campus  versions  and  support  a  wide  variety  of  teaching  methods,  from  lecture-based 
courses  to  team  project-based  courses.  Multiple  media  options  are  available  for  online 
courses;  from  print,  videotapes  distributed  through  the  bookstore,  Internet  Web 
resources,  and  computer-mediated  communication. 

Online  degree  programs,  like  their  campus  counterparts,  require  a  dedicated  service 
environment.  The  overall  service  envmmment  creates  the  stage  for  engaging 
individuals  in  a  way  that  creates  a  memorable  learning  event.  Student  services  are  an 
integral  part  of  the  online  environment,  just  as  it  is  for  on-campus  students. 
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The  systems  and  services  supporting  the  online  student  environment  include 
admissions,  academic  advising,  library  services,  disability  services,  tutorial  services, 
and  technical  support,  evaluating  evidence  of  learning,  and  granting  certification. 

When  online  degree  programs  are  offered  to  primarily  off-campus  students,  the  on- 
campus  services  are  unlikely  to  be  able  to  provide  the  rich  and  responsive  environment 
needed  for  a  successful  online  program.  Most  on-campus  systems  rely  on  the  physical 
campus  and  individual  instructors  to  act  as  the  primary  service  contacts  for  students. 
The  pressure  is  already  on  for  online  instructors  to  facilitate  and  manage  the  important 
role  of  interactivity  and  human  communication  without  being  the  primary  contact  for 
student  technical  support  as  well. 

The  costs  of  a  responsive  online  service  environment  need  to  be  factored  in  to  cost 
comparisons  with  on-campus  counterparts  to  reflect  the  true  costs  of  online  programs  (a 
challenge  the  military  also  faces).  A  totally  online  environment  requires  an  extensive 
technology  infrastructure  and/or  new  strategic  partnerships  that  allow  for  these 
services. 

Keywords:  Costs,  Specialized  Literature 

24.  George,  E.  L.,  Bretl,  D.,  and  Jackson,  G.  (2001),  MOS  92 A  Distance  Learning 
Training  Effectiveness  Analysis,  TRADOC  Analysis  Center-White  Sands  Missile 
Range  (TRAC-WSMR)  (In  Press). 

One  of  the  major  findings  of  this  military  study  was  that  a  key  element  in  the 
effectiveness  of  the  DL  programs  is  structuring  the  courseware  for  the  kind  of  material 
being  taught.  For  example,  thinking  and  leadership  skills  are  not  taught  the  same  way 
manual  skills  are  taught.  In  addition,  effective  courseware  also  must  be  tailored  to 
meet  the  skill  level  and  maturity  of  the  student.  Soldiers  who  had  difficulty  learning 
the  complex  automated  systems  used  in  their  military  occupational  specialty  (MOS  92A) 
benefited  greatly  from  interaction  with  instructors  and  other  students.  This  creates  a 
challenge  for  potential  "instructorless"  training  for  soldiers  who  do  not  have  basic  skills 
and  knowledge  in  the  subject  areas.  Soldiers  with  some  job  experience  did  not  have 
difficulty  with  the  multimedia  learning  approach.  Thus,  teaching  initial  skills  to 
students  requires  different  types  of  instruction  than  teaching  students  with  a 
foundation  in  the  subject  matter.  The  study  found  that  while  the  evidence  is 
overwhelming  that  DL  works,  use  of  multimedia  instruction  for  initial  skills  training 
may  not  be  worth  the  cost,  since  it  has  to  be  constructed  to  provide  the  structure, 
motivation,  and  task  focusing  that  the  instructor  would  provide  in  a  more  traditional 
classroom  setting. 

Keywords:  Benefits,  Costs,  Automation,  Specialized  Literature 

Not  everyone  in  our  society  has  equal  access  to  computers  and 
technology.  The  military  recruits  individuals  from  diverse  cultural, 
ethtiic,  and  economic  groups.  Some  soldiers  may  be  at  a  disadvantage 
with  respect  to  their  computer  experiences  and  background.  Providing 
some  basic  computer  skills  training  initially  may  alleviate  potential 
traitiing  problems  and  frustrations  and,  help  advance  computer  literacy  for 
the  force. 
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25.  Gladieux,  L.  E.,  and  Swail,  S.  (1999),  The  Virtual  University  and  Education 
Opportunity  -  Issues  of  Equity  and  Access  for  the  Next  Generation,  The 
College  Board. 

The  authors  of  the  report,  Lawrence  E.  Gladieux  and  Scott  Swail,  policy  analysts  for 
the  College  Board,  say  people  of  low  income,  African  Americans,  Hispanics,  and  people 
with  less  education  are  less  likely  to  have  access  to  computers  or  online  services  than 
those  with  higher  incomes,  Whites,  Asians,  and  people  with  college  education.  E-mail 
usage  is  higher  among  college  freshmen  at  universities  and  private  4-year  colleges  than 
it  is  at  2-year  and  historically  Black  institutions.  Mr.  Gladieux,  at  a  recent  briefing  in 
Washington,  also  said  educators  should  be  worried  about  the  cost  of  online  education  for 
both  institutions  and  students.  He  cited  studies  showing  that  distance  education 
programs  were  more  costly  for  institutions  to  offer  and  some  colleges  charge  more  for 
online  courses.  The  authors  recommend  making  widespread  access  a  goal  when 
designing  online  courses.  The  report  is  available  online. 

Keywords:  Course  Design,  Automation,  Specialized  Literature 

26.  Hara,  N.,  and  Kling,  R.  (2000),  Students1  Distress  with  a  Web-based 
Distance  Education  Course:  An  Ethnographic  Study  of  Participants; 
Experiences,  Bloomington,  IN,  Center  for  Social  Informatics,  Indiana 
University. 

The  thesis  of  this  report  is  that  the  majority  of  literature  pertaining  to  distance 
education  courses  that  rely  on  the  Internet  as  a  core  communication  medium  tend  to 
emphasize  the  benefits  of  these  courses  and  minimize  the  difficulties  of  "routinely 
providing  high  quality  courses."  There  are  at  least  five  major  kinds  of  literature  about 
asynchronous  distance  education  courses  that  rely  upon  the  Internet  as  a  core 
communication  medium:  (1)  a  specialized  research  literature,  which  includes  journals 
such  as  the  Journal  of  Asynchronous  Learning  and  the  American  Journal  of  Distance 
Education,  research  monographs,  and  specialized  conferences;  (2)  a  practitioner 
literature  whose  audience  is  administrators  and  faculty  who  may  be  involved  in  such 
courses  (e.g.,  the  Chronicle  of  Higher  Education,  written  advice  for  instructors);  (3) 
instructional  materials  for  students  taking  specific  courses  (e.g.,  syllabi,  orientation 
documents,  tip  sheets);  (4)  popular  accounts  of  such  courses  written  for  the  public  (i.e., 
the  mainstream  press);  and  (5)  marketing  descriptions  of  courses  and  degree  programs 
provided  by  educational  institutions  to  their  prospective  students  and  their  parents, 
employers,  and  others. 

This  article  documents  the  difficulties  students  experienced  while  taking  a 
particular  graduate-level  DL  course.  Students  noted  frustration,  a  feeling  of  isolation, 
anxiety,  confusion,  and  panic.  DL  students  commonly  work  alone  at  night  and  resolving 
technical  and  communication  problems  can  be  more  difficult  under  these  conditions 
than  in  a  face-to-face  class  meeting.  Confusion  over  assignments  and  instructor 
expectations  are  also  harder  to  resolve  in  the  distance  mode.  Relying  on  e-mail  as  a 
primary  communication  mode,  some  students  fell  behind  with  reading  and  posting 
e-mail  and  some  felt  overwhelmed  by  the  volume  of  e-mail.  It  is  time-consuming 
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responding  to  e-mail  traffic  as  opposed  to  conversation.  The  instructor  also  commented 
that  at  the  beginning  of  the  semester  she  was  spending  all  day  reading  and  responding 
to  e-mail  messages.  There  can  also  be  a  lag  time  between  students'  queries  and/or  need 
for  assistance/feedback  and  the  instructors'  response.  Students  also  experienced 
frustration  attempting  to  obtain  information  over  the  Web. 

The  authors  note  that  key  issues  regarding  reliable  and  well-timed  communication 
need  to  be  negotiated  within  a  class  so  that  students  and  instructor's  expectations 
regarding  e-mail,  feedback,  technical  support,  etc.  can  be  aligned,  thereby  reducing 
frustration  and  anxiety.  The  reliance  on  text-based  media  also  increases  the 
communicative  complexity  of  constructing  adequately  unambiguous  conversations  and 
instructions.  What  is  the  level  of  detail  and  phrasing  that  will  be  sufficiently  helpful  to 
others? 

Academic  administrators  often  encourage  faculty  to  teach  online  courses  by 
representing  them  as  easy  to  take  on  rather  than  as  a  complex  instructional 
engagement  that  can  require  new  materials  and  new  behaviors.  High  quality 
education,  whether  online  or  face-to-face,  is  neither  cheap  nor  easy. 

Keywords:  Course  Design,  Instructor,  Specialized  Literature 

27.  Harasim,  L.,  Hiltz,  S.  R.,  Teles,  L.,  and  Turoff,  M.  (1995),  Learning 
Networks:  A  Field  Guide  to  Teaching  and  Learning  Online,  Cambridge,  MA, 
MIT  Press. 

Evaluation  of  online  learning  is  multifaceted,  and  learning  competence  is  only  one 
part  of  the  evaluation  need.  Harasim  et  al.,  discuss  both  top-down  or  "summative"  and 
bottom-up  or  "formative"  evaluation  styles.  Rigorous  evaluation  of  learning 
effectiveness  had  been  lacking  in  the  early  1990s  but  should  and  could  be  performed.  A 
multiplicity  of  evidence  beyond  written  exercises  and  test  scores  should  be  considered; 
some  examples  include  participation  by  students  in  class  discussions,  project  work,  and 
individual  and  group  interviews.  The  book  also  offers  a  checklist  for  faculty  to  facilitate 
collaborative  learning,  such  as  do  not  lecture,  be  flexible  and  patient,  monitor  and 
prompt  for  participation,  etc.  A  checklist  for  students  is  also  provided  and  includes  such 
pointers  as  the  learning  curve  can  be  steep,  make  sure  you  have  convenient  access  to 
the  personal  computer  (PC)  you  will  be  using  for  the  class,  and  respond  to  the  ideas  and 
questions  of  other  students  as  well  as  the  instructor. 

Keywords:  Course  Design,  Specialized  Literature 

The  Navy  has  developed  a  training  delivery  assessment  model  to 
identify  existing  classroom  training  courses  that  present  the  best  resource 
saving  opportunities  for  conversion  to  a  DL  format. 
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28.  Hassen,  J.  E.,  and  Wieckhorst,  R.  A.  (1998),  TRADAM:  A  Catalyst  for 
Training  Technology  Infusion,  Modern  Simulation  and  Training . 

This  article  describes  the  Navy's  Training  Delivery  Assessment  Model  (TRADAM), 
which  is  designed  to  assist  training  sponsors,  managers,  course  developers,  and 
instructors  in  selecting  the  most  appropriate  advanced  training  technologies  for  cost 
effective  training  delivery.  The  Chief  of  Naval  Operations'  Office  of  Training 
Technology  initiated  the  development  of  a  model  to  select  the  best  opportunities  for 
improving  training  effectiveness  and  training  efficiency  through  the  application  of 
advanced  technology  in  December  1995.  The  TRADAM  is  intended  to  provide  a  quick, 
but  fairly  accurate,  identification  of  existing  classroom  training  courses  that  present  the 
best  training  resource  saving  opportunities.  The  TRADAM  process  includes  three 
phases.  The  first  phase  performs  a  quick  screening  on  all  training  courses  to  ensure 
that  the  best  candidates  for  advanced  training  delivery  technology  application  are 
examined  first,  to  ensure  that  the  labor  intensive  requirements  and  economic  analyses 
are  first  performed  on  the  courses  with  the  most  potential  for  cost  avoidances.  Phase 
two  is  an  assessment  of  training  delivery  technology  alternatives  that  match  the 
learning  requirements  of  each  candidate  course.  Phase  three  is  the  economic  analysis  of 
training  delivery  technology  alternatives. 

The  initial  screening  process  starts  with  assessing  each  courses'  potential  for 
exportability.  The  exportability  criteria  determine  whether  a  course  could  be  taught  in 
other  than  a  formal  resident  schoolhouse  or  typical  classroom  environment. 

Exportability  criteria  include:  (a)  short  duration  course  (up  to  3  weeks);  (b)  no  special 
training  equipment  required;  (c)  stand-alone  course,  not  part  of  a  training  pipeline;  (d) 
no  face-to-face  interaction  required  between  students.  If  a  course  fails  to  meet  all 
exportability  criteria,  the  TRADAM  subsequently  will  focus  on  course  compression 
opportunities  that  will  result  in  a  reduction  of  personnel,  schoolhouse  infrastructure, 
and  per  diem  costs. 

Keywords:  Benefits,  Costs,  Specialized  Literature 

29.  Hislop,  G.  W.  (2000,  September  17-20,  2000),  Cost  Effectiveness  of  an  Online 
Degree  Program .  Paper  presented  at  the  ALN  Workshop  2000,  Lake  George, 
NY. 

This  paper  discusses  the  cost  effectiveness  of  a  completely  online  graduate  degree 
program  leading  to  a  master  of  science  (MS)  in  Information  Systems  (MSIS)  at  Drexel 
University.  Students  in  the  online  degree  program  move  through  their  courses  in  a 
group,  an  approach  that  made  program  startup  more  manageable  since  all  the  courses 
did  not  need  to  be  available  online  at  once.  The  MSIS  is  a  professional  degree  with  no 
research  option.  Students,  whether  online  or  traditional,  are  typically  part-time 
students  who  work  full-time. 

The  online  MSIS  was  the  first  substantial  effort  in  online  education  at  Drexel 
University.  The  expectation  from  the  start  was  that  the  new  online  degree  program 
would  make  money  by  increasing  the  number  of  students  paying  tuition.  The  online 
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MSIS  could  attract  students  outside  the  Philadelphia  metropolitan  area,  where  most  of 
the  traditional  MSIS  program  students,  comprised  primarily  of  working  professionals, 
live  and  work.  In  addition,  as  Drexel  positions  itself  as  "Philadelphia's  technological 
university,"  supporting  online  degree  programs  fits  the  university  image. 

How  do  the  costs  of  online  education  compare  to  the  costs  of  traditional  education? 

To  begin  to  answer  this  question,  a  framework  for  measuring  online  program  costs  must 
be  established.  This  framework  may  be  difficult  at  first  to  establish  because  university 
accounting  methods  are  not  well  matched  to  this  type  of  cost  analysis.  It  is  difficult  to 
identify  expenses  or  revenue  for  a  particular  degree,  or  course,  or  student,  etc.,  given  the 
fund  accounting  methods  of  most  universities.  This  article  presents  a  cost  allocation 
framework  for  comparing  the  costs  of  the  online  MSIS  degree  program  with  the 
traditional  MSIS  degree  program.  The  results  are  briefly  summarized:  (1)  The  three 
primary  costs  that  differentiate  the  online  program  costs  are  the  cost  of  technical 
support  staff,  the  cost  of  communication  and  server  hosting,  and  the  incentive  pay  being 
paid  to  faculty  members  who  teach  online;  (2)  The  difference  in  cost  of  physical  capital 
increases  the  expense  of  the  traditional  program.  With  all  costs  included,  the  two 
delivery  modes  are  similar  in  operational  cost,  with  the  cost  of  physical  capital 
balancing  out  the  technology  related  costs  of  the  online  program;  (3)  The  focus  of  the 
analysis  presented  in  this  article  was  on  operational  factors  not  course  development  and 
maintenance. 

Keywords:  Costs,  Specialized  Literature 

30.  Hoffman,  D.  L.  and  Novak,  T.  P.  (1998),  Bridging  the  Racial  Divide  on  the 
Internet,  Science. 

This  article  presents  findings  from  a  recent  survey  of  Internet  access.  Income  and 
education  are  the  variables  found  to  most  likely  influence  access  to  and  use  of 
interactive  electronics  media.  The  differences  between  Whites  and  African  Americans 
in  the  US  with  respect  to  computer  access  and  Web  use  was  explored  to  examine 
whether  observed  race  differences  in  access  and  use  can  be  accounted  for  by  differences 
in  income  and  education. 

The  data  for  this  analysis  were  provided  by  Nielsen  Media  Research,  from  the 
Spring  1997  CommerceNet/Nielsen  Internet  Demographic  Study  (IDS),  conducted  from 
December  1996  through  January  1997.  Survey  results  included  that  Whites  were 
significantly  more  likely  than  African  Americans  to  have  a  home  computer  in  their 
household;  while  nearly  twice  as  many  African  Americans  as  Whites  stated  that  they 
planned  to  purchase  a  home  computer  in  the  next  6  months.  More  than  twice  as  many 
Whites  as  African  Americans  had  used  the  Web  in  the  past  week  (1.4  million  +-  0.5 
million  African  Americans  compared  to  20.3  million  +-1.6  million  Whites  used  the  Web 
in  the  past  week). 

With  respect  to  income,  for  households  with  incomes  under  $40,000  Whites  were 
proportionally  twice  as  likely  as  African  Americans  to  own  a  home  computer  and 
slightly  more  likely  to  have  computer  access  at  work.  For  household  incomes  of  $40,000 
or  more,  a  slightly  greater  proportion  of  African  Americans  owned  a  home  computer, 
and  a  significantly  greater  proportion  had  computer  access  at  work. 
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With  respect  to  education,  Whites  were  significantly  more  likely  to  own  a  home 
computer  than  were  African  Americans  regardless  of  educational  level.  However, 
greater  education  corresponded  to  an  increased  likelihood  of  work  computer  access, 
regardless  of  race. 

With  respect  to  students,  race  almost  always  matters.  Students  (including  both  high 
school  and  college  students)  were  more  likely  than  any  other  income  or  educational 
group  to  have  used  the  Web  (presumably  students  have  access  to  the  Web  at  school, 
regardless  of  race).  A  dramatic  difference  between  White  and  African  American 
students  was  in  home  computer  ownership;  whereas  73  percent  of  White  students 
owned  a  home  computer,  only  32  percent  of  African  American  students  owned  one.  The 
analysis  also  showed  that  White  students  were  significantly  more  likely  than  African 
American  students  to  have  used  the  Web,  especially  in  the  past  week.  There  were  no 
differences  in  use  when  students  had  a  computer  at  home.  White  students  lacking  a 
home  computer  were  also  more  than  twice  as  likely  to  access  the  Internet  from  locations 
away  from  home,  such  as  homes  of  friends,  libraries,  or  community  centers  than  African 
American  students. 

Access  translates  into  usage.  The  authors  warn  that  if  a  significant  segment  of  our 
society  is  denied  equal  access  to  the  Internet,  existing  differences  and  opportunities  in 
employment,  housing,  racial  relations,  etc.,  between  Whites  and  African  Americans  may 
be  exacerbated,  with  further  negative  consequences.  They  encourage  the  pursuit  of 
pi'ograms  that  encourage  home  computer  ownership  and  the  adoption  of  inexpensive 
devices  that  enable  Internet  access  through  the  television. 

Keywords:  Course  Design,  Automation,  Specialized  Literature 

31.  Howard,  F.  S.  (1997),  Distance  Learning  Annotated  Bibliography ,  White 
Sands  Missile  Range,  Department  of  the  Army,  TRAC-WSMR,  NM. 

In  1997,  TRAC-WSMR  published  a  DL  annotated  bibliography  summarizing  DL 
reports  in  10  different  categories:  Evaluation  of  DL;  Guidelines  for  planning  and 
implementing  DL;  Computer-based  training;  Video  teletraining  (VTT);  Professional 
education;  Student  interaction  with  instructors,  other  students,  and  technology; 

Reviews  of  DL  literature;  Descriptions  of  some  specific  DL  programs;  Cost  effectiveness; 
and  Miscellaneous.  The  studies  reviewed  covered  the  period  before  1995.  With  one 
exception,  all  of  the  57  evaluation  studies  surveyed  in  this  report  conclude  that  DL  is  as 
effective  as  traditional  instruction.  In  this  and  other  works  covering  several  hundred 
studies,  this  is  the  only  case  where  DL  did  not  fair  at  least  as  well  as  traditional 
instruction,  which  strongly  suggests  this  result  is  a  statistical  fluke. 

Keywords:  Benefits,  Costs,  Course  Design,  Instructor,  Automation,  Specialized 
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32.  (IHETS),  I.  H.  E.  T.  S.  (2000),  Guiding  Principles  for  Faculty  in  Distance 
Learning.  Available:  http://www.ind.net/learntech/facprinc.html. 


A  guiding  principles  subgroup  was  formed  in  2000  with  the  charter  to  "craft  a  set  of 
guidelines  which  define  good  practice  in  DL  and  give  faculty  a  clear  benchmark  for 
designing,  developing,  delivering,  and  assessing  DL  courses  and/or  programs."  Guiding 
principles  and  sub-principles  were  developed  in  the  areas  of  course  design  (careful 
analysis  of  the  target  student  population  lays  the  groundwork  for  building  an  effective 
DL  course),  program  design  (faculty  who  understand  the  wider  contexts  of  their  DL 
programs  are  better  prepared  to  design  courses  that  meet  larger  program  goals),  faculty 
development  (faculty  need  to  seek  out  and  take  advantage  of  appropriate  professional 
development  opportunities  in  technology  and  DL),  course  evaluation  (faculty  will 
develop  procedures  for  the  ongoing  review  of  courses  for  quality,  consistency  with  the 
curriculum,  currency,  and  advancement  of  learning  outcomes),  and  assessment  of 
student  outcomes  (a  systematic  approach  to  assessment  will  be  developed,  etc.).  This 
article  also  contains  a  useful  list  of  other  resources  with  respect  to  guidelines  for 
distance  education. 

Keywords:  Benefits,  Course  Design,  Instructor,  Specialized  Literature 

33.  Jacques,  M.  M.  (1996),  How  to  Find  World  Wide  Web  Distance  Education 
Resources,  University  of  Wisconsin-Extension. 

This  article  should  be  very  helpful  to  those  planning  or  designing  a  DL  website.  The 
author  provides  some  indicators  for  what  makes  an  excellent  website,  and  includes  a  list 
of  some  distance  education  websites  that  the  author  considers  to  demonstrate  the 
qualities  of  an  excellent  DL  website. 

Keywords:  Automation,  Specialized  Literature 

34.  Jewett,  F.  (1999),  Evaluating  the  Benefits  and  Costs  of  Mediated 
Instruction  and  Distributed  Learning,  California  State  University. 

This  2-year  study  funded  by  the  US  Department  of  Education  used  a  set  of  8-10  case 
studies  to  obtain  data  on  the  benefits  and  costs  of  specific  distributed  instructional 
delivery  modes  compared  to  the  benefits  and  costs  of  regular  classroom  delivery.  Modes 
studied  included  interactive  computer  learning  labs  and  networks,  compressed  two-way 
live  video  and  one-way  video/two-way  audio  delivered  by  instructional  television  fixed 
service  (ITFS),  broadcast,  satellite,  or  cable,  and  materials  delivered  via  the  World  Wide 
Web.  Benefit  assessment  was  based  on  (1)  comparisons  of  educational  outcomes 
between  similar  courses  delivered  by  different  means,  (2)  the  potential  to  improve 
course  completion  rates  and  student  retention  rates,  (3)  the  potential  to  improve 
student  access  to  instruction,  (4)  the  potential  for  institutional/curriculum  renewal,  (5) 
other  benefits  identified  relative  to  specific  case  studies.  Cost  comparisons  were  based 
on  estimates  of  the  direct  costs  (both  operating  and  capital  costs)  for  the  delivery  of  the 
specific  form  of  instruction  in  each  case  study. 
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The  case  study  project  determined  that  learning  outcomes  on  every  measured 
criterion  were  at  least  as  good,  and  sometimes  better,  than  on-campus  instruction. 
Distributed  delivery  modes  also  increased  access  to  learning  and  learning  opportunities 
and  opened  up  new  markets  for  universities.  Faculty  and  students  reported  favorably 
to  the  programs,  and  in  some  cases,  class  sizes  could  be  increased  without  loss  of 
learning  or  additional  instructor  workload,  a  cost  savings.  With  sufficient  enrollments, 
DL  is  less  expensive  than  classroom  instruction  and  the  same  benefits,  at  least,  can  be 
derived  at  less  cost. 

Keywords:  Benefits,  Costs,  Specialized  Literature 

35.  Jewett,  F.  (2000),  BRIDGE:  A  Model  for  Comparing  the  Costs  of  Using 
Distance  Instruction  and  Classroom  Instruction,  The  American  Journal  of 
Distance  Education ,  14( 2),  37-47. 

BRIDGE  is  a  computerized  simulation  model  designed  to  estimate  the  costs  of  a 
campus  that  is  offering  all,  or  most,  of  its  instruction  using  electronics-based  distance 
technology.  Because  there  has  been  little  cost  data  for  campuses  in  the  US  where  all  or 
a  substantial  amount  of  the  instruction  is  provided  via  distance  technology,  it  is  difficult 
to  make  cost  comparisons  between  a  traditional  classroom  campus  and  a  campus  that 
offers  the  majority  of  its  instruction  using  distance  technology.  Cost  studies  suggest 
that  economies  of  scale  can  exist  at  the  course  level. 

The  working  hypothesis  underlying  any  valid  cost  comparison  is  that  the  benefits  of 
the  distance  forms  of  instruction  are  at  least  as  good  as  those  of  classroom  instruction. 
Between  1996  and  1998,  the  author  conducted  eight  benefit  cost  case  studies  on  various 
forms  of  television  and  computer-mediated  course  delivery  at  university  campuses  in 
the  US.  The  case  study  comparisons  found  no  significant  difference  between  the 
learning  outcomes  associated  with  classroom  and  mediated  delivery.  This  result  is 
consistent,  according  to  this  article,  with  the  findings  of  Moore  and  Thompson's  1990 
survey,  Russell's  1999  bibliography  of  US  literature,  and  Rumble's  1997  summary  of 
similar  research  outside  the  US. 

Phipps  and  Merisotis  challenged  the  findings  of  no  significant  difference,  who  raised 
questions  about  the  methodologies  used  in  the  various  studies  (What's  the  Difference?). 
The  Phipps  and  Merisotis  critique  has  been  answered  by  Brown  and  Wack  (1999)  and 
by  Ewell  (1999)  in  an  editorial  in  Change  magazine.  Ewell  points  out  that  critiques  of 
individual  studies  ignore  the  fact  that  all  of  the  studies  show  no  significant  difference. 
The  piece  of  the  critique  that  is  missing  is  a  list  of  studies  showing  that  there  is  a 
significant  difference  and  that  the  difference  is  in  favor  of  classroom  instruction.  For 
purposes  of  this  study,  the  benefits  of  mediated  instructional  modes  are  considered  to  be 
at  least  as  good  as  those  of  classroom  instruction  and  that  their  costs  can  be  legitimately 
compared  with  the  costs  of  classroom  instruction. 

The  researchers  separated  the  estimated  cost  of  distance  instruction  related  to 
preparing  and  presenting  the  course  materials  (course-related  costs)  from  those  related 
to  interacting  with  and  evaluating  students  (student-related  costs).  For  classroom 
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instruction,  the  costs  of  these  separate  functions  are  "bundled"  as  part  of  the  cost  of  the 
instructor,  who  has  responsibility  for  preparing  and  presenting  course  materials  as  well 
as  for  student  interaction  and  evaluation.  Approximately  97  percent  of  the  direct  cost  of 
classroom  instruction  is  the  cost  of  the  instructor;  the  imputed  cost  of  the  classroom 
accounts  for  the  rest.  Classroom  costs  rise  in  proportion  to  enrollment.  Instructing 
twice  as  many  students  requires  doubling  the  number  of  course  sections,  which 
essentially  doubles  the  direct  costs. 

The  costs  of  distance  instruction  show  a  different  pattern.  Distance  instruction  has 
a  substantial  start-up,  or  fixed,  cost  that  is  independent  of  enrollment  (studio, 
communication,  materials  preparation,  and  network  costs)  plus  a  variable  cost  that 
depends  on  the  actual  number  of  students  enrolled  (primarily  interaction  and 
evaluation  costs).  As  a  result  of  the  high  start-up  costs  associated  with  distance 
instruction,  classroom  instruction  costs  less  than  distance  instruction  at  relatively  low 
levels  of  course  enrollment.  However,  the  total  costs  of  distance  education  rise  more 
slowly  than  those  of  classroom  instruction.  This  is  because  the  incremental  costs  of 
adding  students  to  a  distance  course  are  less  than  those  of  a  classroom  course,  and  the 
two  cost  functions  ultimately  converge,  and  at  some  level  of  enrollment,  they  intersect. 
This  implies  that  economies  of  scale  do  exist  for  distance  instruction;  average  costs 
decline  as  the  fixed  costs  are  spread  over  larger  course  enrollments.  The  author  points 
out  that  most  US  experience  with  distance  education  has  been  with  relatively  small 
projects  to  date.  These  smaller  distance  education  programs  have  been  operating  at 
enrollment  levels  to  the  left  of  the  crossover  point  and  tend  to  be  more  expensive  than 
traditional  classroom  instruction. 

The  BRIDGE  Cost  Simulation  Model  addresses  the  potential  economies  of  scale  by 
addressing  the  cost  implications  for  a  campus  if  a  substantial  amount  of  its'  instruction 
(25-75  percent)  were  to  be  provided  using  a  given  type  of  distance  instruction.  What 
combinations  of  costs  and  enrollment  levels  for  distance  courses  are  cost  competitive 
with  classroom  courses  when  a  large  percentage  of  instruction  is  provided  via  distance 
courses? 

The  BRIDGE  model  is  an  on-going  project;  at  this  point  the  model's  results  should  be 
considered  illustrative  rather  than  definitive,  cautions  the  author.  Some  results  to  date: 

1.  Television/broadcast  courses  can  be  cost  competitive  with  classroom  delivery 
if  annual  course  enrollments  are  in  the  range  of  150-300.  To  accommodate  this 
enrollment,  a  campus  would  require  full-time  access  to  15-30  channels;  limited  supplies 
of  channels  can  constrain  a  campus's  ability  to  provide  a  broad  range  of  instructional 
programs. 

2.  Asynchronous  network  course  costs  are  more  complex  than  simulating  the 
costs  of  television  courses.  This  type  of  instruction,  in  which  interactive  course 
materials  are  resident  on  a  server  and  are  accessible  to  students  anytime  and  anyplace, 
is  relatively  new  and  cost  data  are  scarcer  than  for  television/broadcast  courses. 
Consequently,  the  BRIDGE  model  allows  the  user  to  literally  design  a  course  by 
inputting  the  activities  the  student  is  expected  to  undertake  during  a  typical  week, 
including  hours  using  interactive  courseware,  e-mail,  and  chat  rooms;  working  with 
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other  students;  interacting  with  an  instructor  or  tutor;  and  taking  exams,  etc.  Course 
production  costs  are  calculated  based  on  faculty  and  media  staff  workload  and  salaries, 
administrative  support,  special  equipment,  communications,  and  other  operating 
expenses.  The  BRIDGE  model  showed  the  crossover  point  in  the  range  of  150-300 
average  annual  course  enrollments  with  an  assumed  course  production  cost  of  $200,000, 
a  5-year  useful  life  for  the  instructional  materials,  and  interaction/evaluation  costs  in 
the  range  of  $150- $250  per  student.  Doubling  course  production  costs  moves  the 
crossover  point  closer  to  400  enrollments. 

3.  Course  sharing,  in  which  two  or  more  campuses  share  the  course  production 
costs  and  all  list  the  shared  course  as  its  own  (each  campus  continues  to  incur  the 
student  interaction  and  evaluation  costs  associated  with  its  own  students)  provides  the 
potential  for  total  course  enrollments  sufficient  to  obtain  the  benefits  of  economies  of 
scale. 

Keywords:  Benefits,  Costs,  Specialized  Litei*ature 

36.  Kashy,  E.,  Thoennessen,  M.,  Tsai,  Y.,  Davis,  N.  E.,  and  Wolfe,  S.  L.,  Using 
Networked  Tools  to  Enhance  Student  Success  Rates  in  Large  Classes. 

This  article  discusses  the  use  of  technology  in  order  to  enhance  student  learning  and 
performance  in  large  classes.  The  key  elements  of  the  2-year  project  included 
implementing  an  active  learning  environment,  identifying  students  at  risk  early,  and 
implementing  a  program  to  mentor  those  students  and  to  assess  the  impact  various 
program  components  have  had  on  success  rates.  The  use  of  technology  has  permitted  a 
reallocation  of  instructors'  and  teaching  assistants'  time,  shifting  it  from  repetitive  jobs 
such  as  grading  and  record  keeping  to  tasks  more  directly  related  to  student 
achievement. 

The  results  of  the  research  program  proved  encouraging.  Technology  can  be 
implemented  in  such  a  way  as  to  provide  students  with  the  opportunity  to  excel,  by 
providing  immediate  feedback,  correction  of  mistakes,  and  the  ability  to  seek  assistance 
in  highly  flexible  ways. 

Keywords:  Benefits,  Course  Design,  Specialized  Literature 

A  Carnegie  Mellon  study  reveals  negative  potential  of  heavy  Internet 
use  on  emotional  well-being. 

37.  Kraut,  R.,  Lundmark,  V.,  Patterson,  M.,  Kiesler,  S.,  Mukopadhyay,  T.,  and 
Scherlis,  W.  (1998),  Internet  Paradox:  A  Social  Technology  that  Reduces  Social 
Involvement  and  Psychological  Well-Being?,  The  American  Psychologist. 

Published  in  the  September  1998  issue  of  The  American  Psychologist,  a  publication 
of  the  American  Psychological  Association,  main  author  Robert  Kraut  was  surprised  to 
find  that  what  is  a  social  technology  has  such  antisocial  consequences.  Heavy  use  of 
e-mail  and  other  communication  services  on  the  Internet  was  associated  with  later 
declines  in  talking  among  family  members,  reductions  in  the  number  of  friends  and 
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acquaintances  they  kept  up  with,  and  increases  in  depression  and  loneliness.  Members 
of  the  research  team  are  part  of  Carnegie  Mellon's  Human-Computer  Interaction 
Institute. 

Keywords:  Course  Design,  Recent  Trends,  Specialized  Literature 

38.  Kulik,  J.  A.  (1994),  Meta-analytic  Studies  of  Findings  on  Computer-based 
Instruction,  In  E.  L.  Baker  and  F.  O'Neil  (Eds.),  Technology  Assessment  in 
Education  and  Training  (pp.  9-33),  Hillsdale,  NJ,  Lawrence  Erlbaum 
Associates,  Inc. 

This  report  was  reviewed  in  TRAC-WSMR's  first  annotated  bibliography  of  DL 
literature  and  is  included  here  as  a  representation  of  a  meta-analytic  study  from  the 
literature  that  concludes  overwhelmingly  that  computer-based  instruction  (CBI)  is  at 
least  as  effective,  and  sometimes  more  effective,  than  traditional  classroom  instruction 
as  measured  by  student  learning  outcomes.  Students  also  learned  in  less  time  with 
CBI.  The  study  summarized  the  major  points,  mostly  favorable,  with  respect  to  CBI 
from  12  separate  meta-analyses. 

Keywords:  Benefits,  Specialized  Literature 

In  the  1980s,  a  number  of  scientists  observed  that  computers  have  been 
growing  exponentially  in  power,  regardless  of  the  type  of  hardware  used. 

In  the  future,  a  $1,000  personal  computer  will  match  the  computing  speed 
and  capacity  of  the  human  brain  by  around  2020.  Futurist  Ray  Kurzweil 
provides  a  fascinating  glimpse  into  the  21st  Century  in  his  book,  "The  Age 
of  Spiritual  Machines. " 

39.  Kurzweil,  R.  (1999),  The  Age  of  Spiritual  Machines:  When  Computers 
Exceed  Human  Intelligence,  New  York,  NY,  Viking  Penguin. 

Among  the  many  salient  points  brought  forth  in  this  fascinating  look  at  the  21st 
century  is  that  the  pace  of  events  is  increasing  dramatically;  thus  is  the  nature  of 
exponential  growth,  that  events  develop  slowly  for  long  periods  of  time,  but  as  one 
"glides  through  the  knee  of  the  curve,  events  erupt  at  an  increasingly  furious  pace."  The 
exponential  growth  of  computing  power  is  used  to  illustrate  the  point.  The  speed  and 
density  of  computation  have  been  doubling  every  3  years  to  every  1  year  toward  the  end 
of  the  20th  century  regardless  of  the  type  of  hardware  used.  Computers  are  about  one 
hundred  million  times  more  powerful  for  the  same  unit  cost  than  they  were  a 
half-century  ago.  As  with  any  phenomenon  of  exponential  growth,  the  increases  are  so 
slow  at  first  as  to  be  practically  unnoticeable.  The  exponential  growth  in  computational 
power  was  barely  noticed  until  the  mid-1960s. 

Keywords:  Benefits,  Costs,  Recent  Trends,  Specialized  Literature 

40.  Lauzon,  A.  C.  (2000),  Distance  Education  and  Diversity:  Are  They 
Compatible?,  The  American  Journal  of  Distance  Education,  14(2),  61-70. 

Market  domination  leads  to  the  eclipsing  of  diversity  in  education,  asserts  Lauzon  in 
this  thought  provoking  article.  Distance  education  is  in  part  a  result  of  the  influences  of 
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economic  globalization.  Distance  education  programs  are  being  increasingly  designed  to 
meet  not  learner  needs  but  market  needs.  Distance  education  is  an  example  of  a  field  of 
inquiry  that  defines  its  evolution  in  terms  of  technological  development.  Are  distance 
education  programs  serving  only  the  interests  of  those  in  power?  The  agenda  of  the 
powerful  may  be  obscured  by  the  presentation  of  technology  as  value-neutral  hardware. 
Educational  institutions  see  technology  as  a  way  of  increasing  their  market  share  while 
at  the  same  time  realizing  efficiencies.  If  the  very  function  of  education  is  to  reduce 
groups  of  persons  to  a  single  cultural  identity,  to  foster  student  conformity  and  to 
facilitate  student  adaptation  of  the  dominant  ideology,  can  distance  education  design  be 
sensitive  to  diversity  and  to  diverse  communities? 

Keywords:  Recent  Trends,  Specialized  Literature 

41.  Lee,  A.  Y.,  Gillan,  D.  J.,  and  Harrison,  C.  L.  (1996),  Assessing  the 
Effectiveness  of  a  Multimedia-based  Lab  for  Upper  Division  Psychology 
Students,  Behavior  Research  Methods,  Instruments,  and  Computers,  28(2), 
295-299. 

This  article  discusses  an  experiment  designed  to  test  whether  a  multimedia-based 
laboratory  could  increase  learning  on  two  different  topics,  as  compared  with  a 
recitation- style  lecture.  Sixty-four  undergraduate  students  at  New  Mexico  State 
University  (NMSU)  who  were  taking  an  upper  division  course  on  learning  participated 
in  the  experiment. 

A  within-subjects  design  was  used.  For  the  first  half  of  the  semester,  half  of  the 
students  used  the  operant  conditioning  software  and  half  listened  to  a  recitation-style 
lecture  on  the  same  topic.  In  the  second  half  of  the  semester,  the  students  who  heard 
the  recitation  in  the  first  half  used  software  on  eyewitness  memory  and  those  who  used 
operant  conditioning  software  listened  to  a  recitation  on  eyewitness  memory. 

Three  types  of  data  were  collected:  pretest  and  posttest  scores  and  usability 
questionnaire  scores.  The  students  in  both  conditions  (computer  and  recitation) 
improved  from  pretest  to  posttest.  A  trend  in  the  data  for  the  operant  conditioning 
portion  of  the  class  indicates  that  for  the  recitation,  the  lecture  helped  the  subjects 
scoring  low  on  the  pretest  more  than  those  scoring  high  on  the  pretest;  however,  for 
those  using  the  computer,  multimedia  helped  the  subjects  equally  across  the  range  of 
pretest  scores.  This  result,  according  to  the  authors,  may  have  been  due  to  the  large 
number  of  conceptual  questions  on  the  pre-  and  posttests. 

For  the  eyewitness  memory  portion,  the  data  reveals  an  opposite  result;  for  those 
using  the  computer,  the  subjects  who  scored  lower  on  the  pretest  improved  more  than 
did  those  who  scored  high  on  the  pretest.  For  those  students  in  recitation,  the  students 
were  helped  equally  across  the  range  of  pretest  scores.  In  some  cases,  replacing 
recitations  with  a  multimedia  lab  may  benefit  students  with  low  initial  knowledge,  but, 
in  other  cases,  a  recitation  may  be  better.  These  results  imply  that  each  new  piece  of 
multimedia  software  may  need  its  own  evaluation  in  order  to  determine  its  effectiveness 
unless  a  set  of  empirically  derived  multimedia-based  training  guidelines  are  developed. 
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For  training  students  with  less  initial  knowledge  in  the  topic,  multimedia  training 
software  may  need  to  include  more  aid,  such  as  goal-directedness,  feedback  on  activities, 
and  connections  with  previous  knowledge.  The  authors  conclude  that  the  area  of 
multimedia  would  benefit  from  a  set  of  tested  guidelines  such  as  those  created  for 
intelligent  tutoring  systems. 

Keywords:  Benefits,  Course  Design,  Automation,  Specialized  Literature 

42.  Levinson,  D.  P.  (1998),  The  Book  on  the  Book:  A  Prognosis  for  the  Page  in 
the  Digital  Age,  Analog  Science  Fiction  and  Fact,  Vol.  CXVIIKfi),  24-31. 

This  interesting  article  addresses  the  history  and  future  of  books;  can  the  book 
survive  the  digital  revolution  and  what  characteristics  of  books  will  help  them  endure 
over  time? 

Keywords:  Recent  Trends,  Popular  Literature 

43.  Lewis,  L.,  Snow,  K.,  Farris,  E.,  Levin,  D.,  and  Green,  B.  (1999),  Distance 
Education  at  Postsecondary  Education  Institutions  1997-98,  Washington,  DC, 
US  Department  of  Education,  National  Center  for  Education  Statistics  (NCES). 

In  the  1997-1998  academic  year,  approximately  one-third  of  the  5,000  2-  and  4-year 
postsecondary  institutions  in  the  US  offered  distance  education  courses,  while  another 
fifth  planned  to  do  so.  Overall,  US  higher  education  institutions  reported  1,661,100 
students  enrolled  in  distance  education  courses  in  1998.  The  most  popular  delivery 
technologies  were  asynchronous  Internet  instruction  (58  percent),  two-way  interactive 
video  (54  percent),  and  one-way  prerecorded  video  (47  percent).  While  a  snapshot  in 
time,  these  statistics  illustrate  the  rapid  expansion  of  DL  in  our  society. 

Keywords:  Recent  Trends,  Specialized  Literature 

Access  to  university  training  during  a  tour  of  duty  aboard  a  Navy 
aircraft  carrier  is  now  possible  due  to  a  pilot  project  sponsored  by  the 
naval  education  and  training  offices  in  Pensacola,  FL. 

44.  Mendels,  P.  (1999,  January  20,  1999),  Distance  Learning  Even  Reaches 
Ships  at  Sea,  New  York  Times. 

A  new  Navy  program,  begun  in  the  fall  of  1997,  allows  shipboard  officers  and 
enlisted  men  and  women  to  pursue  graduate  degrees  while  on  board  a  nuclear-powered 
US  Navy  aircraft  carrier  now  somewhere  in  the  Persian  Gulf.  The  program  was 
introduced  as  a  pilot  project,  according  to  Saundra  K.  Drummer,  who  directs  education 
and  training  strategies  at  the  naval  education  and  training  offices  in  Pensacola,  FL. 

Now  available  on  two  aircraft  carriers,  the  program  eventually  could  be  rolled  out  to  all 
12  in  the  Navy.  The  program  brings  university  classes  from  three  different  southern 
universities,  Georgia  College  and  State  University,  in  Milledgeville,  GA,  Old  Dominion 
University  in  Norfolk,  VA,  and  Troy  State  University  in  Troy,  AL,  to  Navy  personnel  on 
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board  ship  via  computers,  satellite  transmissions,  and  video  conferencing.  Two-way 
teleconferencing  technology  beams  the  voice  and  moving  image  of  the  professor  and 
Milledgeville  (for  example)  students  live  to  the  ship-based  class.  Likewise,  the  land- 
based  participants  can  see  and  hear  their  Navy  counterparts.  Outside  of  class,  the 
Internet  provides  the  mechanism  by  which  assignments,  corrections,  and  revisions  are 
transferred  between  students  and  the  professor  via  e-mail.  Classmates  exchange 
thoughts  through  a  Web  site  that  posts  each  student's  picture  next  to  his  or  her 
comments. 

Scheduling  the  class  time  becomes  a  challenge  as  the  Navy  ships  move  and 
maneuver.  The  professor  in  this  write-up,  Professor  Edmonson  of  Georgia  College,  is 
willing  to  change  the  hours  of  the  class  frequently  to  ensure  that  the  class  reaches  his 
Navy  students  at  6  am,  the  time  best  suited  for  them,  regardless  of  the  ships'  position. 
This  means  even  teaching  class  at  midnight.  Technical  problems  are  almost  inevitable 
with  ship  movements,  waves,  and  bad  weather.  Also,  machinery  cannot  completely 
duplicate  the  spontaneity  of  real  human  contact,  and  conversation  is  more  deliberate 
and  constrained  at  times,  according  to  Edmonson.  On  the  other  hand,  the  technology 
requires  students  to  focus  more  on  the  matter  at  hand  and  may  encourage  them  to  be 
more  articulate. 
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45.  Metes,  G.  S.,  Ph.D.,  and  Thomas,  C.,  Ph.D.  (1998),  Digital  Summer  Session: 
Evaluation  Report,  Ed  Journal ,  12(6),  21-30. 

The  California  State  University  (CSU)  Digital  Summer  Session  (DSS)  featured  43 
online  courses  delivered  to  541  students  by  11  campuses  from  June  through  September 
1997.  The  courses  were  delivered  primarily  through  electronic  publishing,  dialogue- 
support  (asynchronous  conferencing  and  real  time  chat),  and  e-mail.  Some  courses  were 
augmented  with  face-to-face  class  meetings  or  labs,  videotapes,  CD-ROM,  and  server- 
based  applications.  All  courses  were  specifically  designed  for  electronic  delivery.  A 
variety  on  online  support  processes  were  instituted,  ranging  from  Web-based  course 
descriptions,  online  registration,  faculty  office  hours,  library  support,  and  course 
evaluation.  The  objectives  of  the  DSS  evaluation,  sponsored  by  the  CSU  chancellor's 
office,  were  to  learn  what  works  and  what  does  not  work  in  online  learning 
environments,  and  to  formally  articulate  that  learning  for  interested  members  of  the 
CSU  community.  Surveys  and  questionnaires  were  administered  to  students,  faculty, 
academic  executives,  technology  support  staff,  academic  support  staff,  and  student 
services  support  staff.  Evaluation  findings  included: 

Faculty  Feedback  -  The  overwhelming  majority  thought  DSS  was  a  good  idea. 

Concerns  expressed  included  continuity  of  the  program,  sources  of  funding  for 
faculty  support  to  develop  online  courses,  financial  aid,  transfer  of  credit,  library 
privileges,  and  fee  structures,  particularly  with  respect  to  students  from  one  campus 
taking  courses  at  another.  Faculty  recommendations  included  that  CSU  should  invest 
in  a  few  larger,  collaborative  multi-year  projects  to  explore  DL  opportunities,  instead  of 
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a  lot  of  small  one-shot  projects,  CSU  committees  should  focus  on  the  most  critical 
policies  essential  to  supporting  DL  and  that  student  records  should  not  indicate  that  a 
course  was  face-to-face  or  online. 

Student  Feedback  -  Most  students,  especially  adult  learners,  felt  that  the  Web- 
based  offerings  were  as  good  as,  or  better  than,  face-to-face  course  meetings.  Many  saw 
distinct  advantages  to  online  courses,  such  as  being  able  to  take  courses  at  all,  with 
working  full  time  or  not  being  geographically  close  to  campus. 

Evaluators  Feedback  -  Web-based  courses  are  proliferating  at  all  campuses 
exponentially.  This  expansion  is  not  (nor  should  it  be)  centrally  manageable.  Some 
campuses  have  more  expertise  in  some  areas  than  others  and  are  eager  to  share  that 
expertise.  Whole  programs,  not  just  courses,  should  be  candidates  for  DL  delivery.  The 
capability  to  produce  and  deliver  at  campuses  is  barely  an  issue  now  and  will  not  be  an 
issue  in  about  a  year  or  so. 

Keywords:  Benefits,  Course  Design,  Instructor,  Specialized  Literature 

The  Deputy  Secretary  of  Defense  tasked  the  Services  on  27  October 
1994  to  review  DL  technologies  and  to  submit  plans  by  31  July  1995  for 
using  them  to  tram  their  reserve  component  (RC)  personnel.  This  study 
provides  a  review  of  these  Program  Decision  Memorandum  (PDM)  directed 
Service  Plans,  summarizes  the  major  findings  from  the  review,  and  offers 
recommendations  based  on  the  findings. 

46.  Metzko,  J.,  Redding,  G.  A.,  and  Fletcher,  J.  D.  (1996),  Distance  Learning 
and  the  Reserve  Components,  Washington,  DC,  Institute  for  Defense  Analyses 
(IDA). 

This  report  provides  a  summary  of  service  official  plans  to  adopt  DL  technologies  for 
training.  The  report  examines  the  potential  of  utilizing  DL  for  RC  training.  Over  $55.3 
million  dollars  were  spent  on  travel  and  per  diem  in  FY95  to  support  the  training  of  RC 
students  in  centralized  training  facilities.  The  RC  component  has  a  far  higher 
percentage  of  non-MOS  qualified  personnel  for  their  duty  positions  (39  percent) 
compared  to  the  active  Army  (AC)  (only  4  percent  are  non-MOS  qualified).  Only  41 
percent  of  the  Army  RC  personnel  who  were  not  MOS  qualified  were  slated  to  attend 
resident  training  in  FY95  due  to  lack  of  funding.  Thus,  one  of  the  principle  findings  of 
this  study  was  that  "the  economic  value  of  increased  training  accessibility  and  the 
resulting  increased  readiness  available  from  more  comprehensive  use  of  DL 
technologies  is  difficult  to  estimate,  but  could  be  greater  than  the  training  cost  savings. 
It  may  prove  to  be  the  primary  payoff  from  systematic  adoption  of  DL  technologies  for 
RC  training." 


47 


Other  principle  findings  from  this  review  of  the  service  DL  plans  include: 

•  More  than  300  studies  have  demonstrated  the  efficacy  of  DL  technologies.  DL  is 
as  effective  as  conventional  resident  instruction  in  achieving  a  wide  range  of 
instructional  objectives. 

•  DL  technology  applications  could  be  used  to  train  AC  personnel  (1.6  million)  and 
Department  of  Defense  (DoD)  civilians  (900,000)  as  well  as  RC  personnel. 

•  DL  technology  could  be  used  to  train  the  nearly  46,000  Army  and  Navy  RC 
personnel  who  required  training  in  FY95,  in  contrast  to  the  20,000  RC  personnel 
who  attended  resident  training  in  that  year. 

•  Many  cost  savings  can  be  potentially  realized  from  use  of  DL  technologies, 
including  fiscal  savings  in  travel  and  per  diem,  reductions  in  time  to  train, 
training  material  costs,  and  instructor  costs. 

•  Converting  to  DL  requires  sizable  up-front  investments.  Cost  estimates  to 
convert  conventional  schoolhouse  instruction  to  DL  courseware  vary  widely. 
Service  investment  cost  estimates  for  RC  DL  applications  are  sizable,  ranging 
from  about  $500  million  to  $1,800  million. 

•  Most  service  expenditures  for  DL  have  been  in  hardware  rather  than 
courseware,  a  practice  that  limits  the  utilization  of  DL  systems. 

•  Data  gaps  remain  in  all  service  plans  for  using  DL  in  RC  training.  Service 
budgets  provide  no  budget  line  items  or  budget  exhibits  under  which  funding  for 
DL  can  be  uniformly  tracked  and  managed.  Service  plans  do  not  identify  how 
implementing  DL  would  affect  the  present  schoolhouse  infrastructure  and  costs; 
marginal  costs  of  the  schoolhouse  infrastructure  (instructors,  training 
developers,  training  equipment,  course  maintenance,  school  operations,  etc.)  are 
not  available. 

The  study  recommends  examining  the  implementation  of  DL  technologies  from  a 
total  force  context  given  that,  with  few  exceptions,  the  course  content  is  the  same  for  AC 
and  RC  personnel  in  the  same  job  occupations.  This  could  make  DL  technology 
investment  more  economically  attractive.  The  report  also  recommends  preparing  an 
action  plan  to  help  expeditiously  move  the  plan  forward  for  implementation. 

Keywords:  Benefits,  Costs,  Specialized  Literature 

47.  Miller,  G.  E.  (2000,  17-20  September  2000),  Penn  State's  World  Campus:  A 
Case  Study  in  Achieving  Cost  Efficiencies  in  ALN.  Paper  presented  at  the  ALN 
Workshop  2000:  Learning  Effectiveness,  Faculty  Satisfaction,  and  Cost 
Effectiveness,  Lake  George,  NY. 

The  World  Campus  was  inaugurated  in  1998  by  the  Pennsylvania  State  University 
as  a  major  initiative  to  use  the  Internet  and  related  technologies  to  extend 
Pennsylvania  State  programs  to  students  nationally  and  internationally.  Its  5-year  goal 
is  to  offer  up  to  30  certificate  and  degree  programs,  comprising  up  to  300  individual 
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courses,  that  will  generate  10,000  enrollments  per  year.  While  Pennsylvania  State  is  a 
land  grant  university,  classified  as  a  comprehensive  research  extensive  institution  and 
operates  as  a  nonprofit  corporation,  the  World  Campus  is  expected  to  generate  net 
income  after  expenses,  and  has  been  charged  with  the  goal  of  being  entirely  self- 
sufficient  by  the  end  of  its  first  5  years  of  operation.  Thus,  achieving  cost  effectiveness 
and  cost  efficiencies  are  high  priorities  of  the  World  Campus.  The  World  Campus  is 
focusing  on  three  areas  for  achieving  cost  efficiencies:  scope,  scale,  and  streamlining 
With  respect  to  scope,  the  World  Campus  is  on  target  toward  its  5-year  goal  of  having  30 
programs  available  by  2002-2003.  Administrative  overhead  has  been  reduced  by  about 
50  percent.  With  regard  to  scale,  average  course  enrollments  have  grown  from  14  to  28 
in  just  over  2  years.  The  goal  is  for  average  credit  enrollments  to  increase  to  around  50 
by  the  end  of  the  first  5  years  in  operation.  Several  important  efforts  have  also  been 
made  to  streamline  processes  in  order  to  create  capacity  and  reduce  the  unit  cost  of  a 
process.  Pennsylvania  State  utilizes  the  continuous  quality  improvement  (CQI)  process 
to  streamline  processes. 

Keywords:  Costs,  Specialized  Literature 

48.  Moore,  M.  G.  (2000),  Editorial  -  Technology-Driven  Change:  Where  Does  it 
Leave  the  Faculty?,  The  American  Journal  of  Distance  Education,  14(1),  1-6. 

The  roles  of  faculty  may  soon  be  changing,  driven  by  economies  of  large-scale 
production  and  division  of  labor  principles.  Faculty  roles  may  be  split  into  professions  of 
content  development,  facilitating,  and  media  design  and  production.  Blending 
traditional  staffing  structures  and  industrial  delivery  methods  is  not  a  likely  scenario. 
However,  both  can  coexist,  with  some  institutions  providing  the  conventional,  face-to- 
face,  labor  intensive,  largely  tutorial  methods  while  other  institutions  specialize  in 
industrial-type  distance  education.  We  are  entering  a  critical  period,  the  author  states, 
and  it  would  be  unfortunate  if  faculty  were  not  centrally  involved  in  the  decisions  that 
will  determine  the  future  direction  of  higher  education. 

An  interesting  observation  with  respect  to  this  editorial  is  the  discussion  centering 
around  the  price  advantages  that  accrue  to  distance  education  programs  that  benefit 
from  the  economies  of  large-scale  production.  According  to  this  article,  competition  from 
high-quality,  dedicated  distance  education  systems  is  just  around  the  corner  as  many 
universities  are  jumping  in  the  DL  arena.  The  economies  of  large-scale  production 
combined  with  cost  savings  from  travel  reduction  and  less  per  diem  could  be 
substantial  for  the  Army. 

Keywords:  Costs,  Instructor,  Specialized  Literature 

49.  Murray,  J.  H.  (1997),  Hamlet  on  the  Holodeck:  The  Future  of  Narrative  in 
Cyberspace,  Cambridge,  MA,  The  MIT  Press,  Massachusetts  Institute  of 
Technology. 

Multimedia  pioneer  Janet  Murray  is  a  senior  research  scientist  at  the 
Massachusetts  Institute  of  Technology  (MIT).  She  directs  MIT's  Program  in  Advanced 
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Interactive  Narrative  Technology  (PAINT)  and  teaches  a  course  in  interactive  narrative 
In  this  interesting  book  about  multimedia  tools  and  techniques,  Murray  describes  the 
tools  that  the  "cyberbards"  of  the  21st  century  will  need,  as  she  considers  the  profound 
effect  of  the  computer  as  a  medium  of  presentation.  One  unexpected  but  powerful  effect 
is  that  sound  is  beginning  to  supercede  the  print  library  as  a  focus  of  research,  and  the 
resurgence  of  the  importance  of  storytelling,  one  of  the  oldest  forms  of  understanding 
and  transmitting  knowledge.  Murray’s  book  also  offers  examples  of  the  kinds  of 
changes  that  widespread  access  to  broadband  digital  environments  might  bring  us. 

Keywords:  Benefits,  Recent  Trends,  Specialized  Literature 

50.  Parchman,  S.  W.,  Ellis,  J.  A.,  Christinaz,  D.,  and  Vogel,  M.  (2000),  An 
Evaluation  of  Three  Computer-Based  Instructional  Strategies  in  Basic 
Electricity  and  Electronics  Training,  Military  Psychology,  12(1),  73-87. 

This  study  examines  whether  CBI  approaches  can  be  applied  to  the  training  of 
complex  technical  skills.  Performed  by  the  Navy  Personnel  Research  and  Development 
Center  in  San  Diego,  the  study  evaluated  four  alternative  instructional  strategies  for 
teaching  complex  technical  skills  in  basic  electronics.  The  four  strategies  were: 
computer-based  drill  and  practice  (CBDP),  enhanced  computer-based  instruction 
(ECBI),  a  computer-based  adventure  game  (GAME),  and  the  existing  classroom 
instruction  (Cl).  At  the  completion  of  instruction,  trainees  were  administered  a 
schoolhouse  comprehensive  test,  a  specially  designed  cognitive  skills  test,  and  a 
motivation  questionnaii'e. 

Basic  electricity  and  electronics  (BE/E)  fundamentals  are  included  in  the  programs 
of  instruction  for  many  different  careers  in  equipment  and  systems  maintenance. 
Historically  the  BE/E  material  has  proved  difficult  for  students  to  learn  and  has 
resulted  in  high  setback  and  attrition  rates.  In  general,  the  CBDP  and  ECBI  groups 
outperformed  the  Cl  and  GAME  groups  on  all  measures. 

Keywords:  Benefits,  Course  Design,  Specialized  Literature 

The  amount  of  information  that  now  reaches  a  soldier  on  the  battlefield 
is  staggering  compared,  to  even  the  recent  past .  The  ability  to  process  this 
information  and  determine  what  is  actually  important  to  the  decision  at 
hand,  as  well  as  the  ability  to  operate  complex  C2  automated  systems  are 
two  skills  the  officer  in  the  field,  must  now  have .  The  Marine  Corps' 

Combat  Decision  Range,  a  computer  simulation  program,  is  designed  to 
train  troops  to  quickly  sift  through  large  amounts  of  data  and  to  make 
rapid  tactical  decisions  under  stress. 

51.  Peters,  K.  M.  (1999),  Split  Decision,  Government  Executive ,  October  1999, 
43-48. 

This  article  examines  the  impact  of  new  technology  and  information  distribution; 
while  vastly  more  information  reaches  the  soldier  on  the  battlefield  by  means  of  new 
technologies  such  as  palm  top  computers,  the  human  thought  process  remains 
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unchanged.  With  the  massive  amount  of  data  now  available  to  soldiers,  the  problem 
facing  commanders  becomes  knowing  what  to  know.  The  lack  of  attention  to  the  human 
side  of  information  processing  in  favor  of  focusing  on  the  technologies  may  have  been  a 
factor  in  the  case  of  Captain  Will  Rogers  of  the  Navy  cruiser  USS  Vincennes,  whose 
decision  in  July  1988  to  shoot  down  Iran  Air  Flight  655,  which  he  mistakenly  believed  to 
be  an  Iranian  fighter  jet,  resulted  in  the  deaths  of  290  civilians.  Captain  Rogers  based 
his  decision  under  pressure  on  the  Iranian  jet's  course,  speed,  altitude,  and  radio 
broadcasts,  just  a  fraction  of  the  information  available  to  him  from  the  ship's  Aegis 
weapons  system.  Another  captain  aboard  the  USS  Sides,  operating  also  in  the  Gulf  at 
the  time,  concluded  accurately  that  Flight  655  was  a  commercial  jet.  The  different 
conclusions  of  the  two  captains  points  out  that  the  military  would  do  well  to  learn  more 
about  human  decision  making  using  the  new  high-tech  tools.  With  the  new  battlefield 
digitization  programs,  the  Army  hopes  to  achieve  improved  situational  awareness 
thereby  accomplishing  missions  faster  with  fewer  casualties.  However,  simply  getting 
more  data  to  people  is  just  the  beginning.  The  data  must  be  turned  into  information 
which  in  then  interpreted  into  knowledge.  The  soldier  must  be  able  to  interpret  data 
quickly,  at  a  glance. 

Since  the  Army  began  experimenting  with  digitization  in  1994,  researchers  from  the 
Central  Technical  Support  Facility  (CTSF)  at  Fort  Hood  have  learned  that  how 
information  is  conveyed  is  as  important  as  what  information  is  conveyed.  Studies 
revealed  that  information  needed  to  be  displayed  on  large  screens  at  the  battlefield 
headquarters.  Commanders  made  decisions  on  visually  seeing  how  things  are 
presented.  Second,  data  from  different  systems  needed  to  be  integrated  to  make  it 
useful  for  commanders,  e.g.,  staff  officers  had  to  be  able  to  synchronize  data  arriving 
through  intelligence  and  logistics  channels  with  data  arriving  through  combat  system 
channels.  In  addition,  soldiers  have  to  develop  a  higher-than-average  level  of  computer 
literacy  to  use  and  maintain  the  digitized  systems  effectively.  There  is  a  constant  need 
for  training  and  for  maintenance  of  this  highly  technical  infrastructure. 

Like  the  Army,  the  Marine  Corps  has  been  experimenting  with  technology  to 
improve  situational  awareness  on  the  battlefield.  Research  at  the  Marine  Corps 
Warfighting  Laboratory  in  Quantico,  VA  has  evolved  into  the  Combat  Decision  Range,  a 
computer  simulation  program  designed  to  train  troops  to  quickly  sift  through  large 
amounts  of  data  and  to  make  rapid  tactical  decisions  under  stress.  With  the  simulator, 
a  Marine  has  to  make  about  50  to  60  tactical  decisions  within  30  minutes.  Immediately 
following  the  range  training,  Marines  are  debriefed  on  the  decisions  they  made,  why 
they  made  them,  and  whether  they  were  the  right  decisions. 

Keywords:  Benefits,  Automation,  Practitioner  Literature 

Critics  of  DL  have  identified  shortcomings  in  the  DL  literature  to  date, 
and  point  out  challenges,  as  well  as  opportunities,  for  future  research 
efforts. 
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52.  Phipps,  R.  A.,  and  Merisotis,  J.  P.  (1999),  What's  the  Difference?  A  Review 
of  Contemporary  Research  on  the  Effectiveness  of  Distance  Learning  in 
Higher  Education,  Washington,  DC,  The  Institute  for  Higher  Education  Policy. 

A  critical  review  of  distance  education  studies,  this  analysis  identified  some  key 
shortcomings  in  the  DL  research  to  date.  Findings:  (1)  Much  of  the  research  does  not 
control  for  extraneous  variables  and  therefore  cannot  show  cause  and  effect.  (2)  Most  of 
the  studies  do  not  use  randomly  selected  subjects.  (3)  The  validity  and  reliability  of  the 
instruments  used  to  measure  student  outcomes  and  attitudes  are  questionable.,  (4) 

Many  studies  do  not  adequately  control  for  the  feelings  and  attitudes  of  the  students 
and  faculty  -  what  the  educational  research  refers  to  as  "reactive  effects."  (5)  The 
research  has  tended  to  emphasize  student  outcomes  for  individual  courses  rather  than 
for  a  total  academic  program.  (6)  The  research  does  not  take  into  account  differences 
among  students.  (7)  The  research  does  not  adequately  explain  why  the  dropout  rates  of 
distance  learners  are  higher.  (8)  The  research  does  not  take  into  consideration  how  the 
different  learning  styles  of  students  relate  to  the  use  of  particular  technologies.  (9)  The 
research  does  not  adequately  address  the  effectiveness  of  digital  libraries,  etc. 

Phipps  and  Merisotis  conclude  that  the  research  on  DL  and  its  effectiveness  has 
placed  more  emphasis  on  the  "utopian"  possibilities  of  the  technology  and  its  potential 
to  do  as  well  as  CBI  than  on  practical  issues.  Questions  that  need  to  be  asked  include 
"What  is  the  quality  of  the  access  to  learning  in  the  DL  context?"  "What  are  the  best 
ways  to  participate  in  asynchronous  communication?"  "Does  the  student  have  the 
necessary  skills  to  use  the  technology?"  Technology,  asserts  the  authors,  cannot  replace 
the  human  touch  in  education.  Technology  is  not  nearly  as  important  in  ascertaining 
student  learning  outcomes  as  other  factors  such  as  student  motivation,  the  quality  of 
the  instructor  as  well  as  the  course  of  instruction,  and  individual  learner 
characteristics. 
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53.  Policy,  T.  I.  f.  H.  E.  (2000),  Quality  on  the  Line:  Benchmarks  for  Success  in 
Internet-Based  Distance  Education ,  The  Institute  for  Higher  Education  Policy. 

This  case  study  project  was  undertaken  to  compile  a  set  of  benchmarks  that  could  be 
used  consistently  to  evaluate  DL  courses  and,  in  particular,  Internet-based  courses.  A 
comprehensive  literature  search  was  conducted  to  compile  benchmarks  recommended 
by  other  organizations  and  groups  and  in  various  articles  and  publications.  Second, 
institutions  that  have  substantial  experience  in  distance  education  were  identified. 

Third,  these  institutions  were  visited  by  institute  staff  to  assess  the  degree  to  which  the 
campuses  incorporated  the  benchmarks  in  their  Internet-based  DL  courses  and 
programs.  The  six  institutions  participating  in  the  study  were:  Brevard  Community 
College,  Regents  College,  the  University  of  Illinois  at  Urbana-Champaign,  the 
University  of  Maryland  University  College,  Utah  State  University,  and  Weber  State 
University 
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The  final  outcome  of  the  study  is  a  list  of  24  benchmarks  that  can  be  applied  to 
measure  the  quality  of  Internet-based  distance  education.  The  benchmarks  fall  into 
seven  categories:  institutional  support,  course  development,  teaching/learning,  course 
structure,  student  support,  faculty  support,  and  evaluation  and  assessment 
benchmarks. 

Institutional  Support  Benchmarks 

•  A  documented  technology  plan  that  includes  electronic  security  measures  is  in 
place  and  operational  to  ensure  both  quality  standards  and  the  integrity  and 
validity  of  information. 

•  The  reliability  of  the  technology  delivery  system  is  as  failsafe  as  possible. 

•  A  centralized  system  provides  support  for  building  and  maintaining  the  distance 
education  infrastructure. 

Course  Development  Benchmarks 

•  Guidelines  regarding  minimum  standards  are  used  for  course  development, 
design,  and  delivery,  while  learning  outcomes  -  not  the  availability  of  existing 
technology  —  determine  the  technology  being  used  to  deliver  course  content. 

•  Instructional  materials  are  reviewed  periodically  to  ensure  they  meet  program 
standards. 

•  Courses  are  designed  to  require  students  to  engage  themselves  in  analysis, 
synthesis,  and  evaluation  as  part  of  their  course  and  program  requirements. 

Technology/Learning  Benchmarks 

•  Student  interaction  with  faculty  and  other  students  is  an  essential  characteristic 
and  is  facilitated  through  a  variety  of  ways,  including  voice-mail  and/or  e-mail. 

•  Feedback  to  student  assignments  and  questions  is  constructive  and  provided  in  a 
timely  manner. 

•  Students  are  instructed  in  the  proper  methods  of  effective  research,  including 
assessment  of  the  validity  of  resources. 

Course  Structure  Benchmarks 

•  Before  starting  an  online  program,  students  are  advised  about  the  program  to 
determine  if  they  possess  the  self-motivation  and  commitment  to  learn  at  a 
distance  and  that  they  have  access  to  the  minimal  technology  required  by  the 
course  design. 

•  Students  are  provided  with  supplemental  course  information  that  outlines  course 
objectives,  concepts,  and  ideas,  and  learning  outcomes  for  each  course  are 
summarized  in  a  clearly  written  statement. 

•  Students  have  access  to  sufficient  library  resources  that  may  include  a  "virtual 
library"  accessible  through  the  World  Wide  Web. 

•  Faculty  and  students  agree  upon  expectations  regarding  times  for  student 
assignment  completion  and  faculty  response. 
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Student  Support  Benchmarks 

•  Students  receive  information  about  programs,  including  admission  requirements, 
tuition  and  fees,  books  and  supplies,  technical  and  proctoring  requirements,  and 
student  support  services. 

•  Students  are  provided  with  hands-on-training  and  information  to  aid  them  in 
securing  material  through  electronic  databases,  interlibrary  loans,  government 
archives,  news  services,  and  other  sources. 

•  Throughout  the  duration  of  the  course/program,  students  have  access  to 
technical  assistance,  including  detailed  instructions  regarding  the  electronic 
media  used,  practice  sessions  before  to  the  beginning  of  the  course,  and 
convenient  access  to  technical  support  staff. 

•  Questions  directed  to  student  service  personnel  are  answered  accurately  and 
quickly,  with  a  structured  system  in  place  to  address  student  complaints. 

Faculty  Support  Benchmarks 

•  Technical  assistance  in  course  development  is  available  to  faculty,  who  are 
encouraged  to  use  it. 

•  Faculty  members  are  assisted  in  the  transition  from  classroom  teaching  to  online 
instruction  and  are  assessed  during  the  process. 

•  Instructor  training  and  assistance,  including  peer  mentoring,  continues  through 
the  progression  of  the  online  course. 

•  Faculty  members  are  provided  with  written  resources  to  deal  with  issues  arising 
from  student  use  of  electronically-accessed  data. 

Evaluation  and  Assessment  Benchmarks 

•  The  program's  educational  effectiveness  and  teaching/learning  process  is 
assessed  through  an  evaluation  process  that  uses  several  methods  and  applies 
specific  standards. 

•  Data  on  enrollment,  costs,  and  successful/innovative  uses  of  technology  are  used 
to  evaluate  program  effectiveness. 

•  Intended  learning  outcomes  are  reviewed  regularly  to  ensure  clarity,  utility,  and 
appropriateness. 

Keywords:  Benefits,  Costs,  Course  Design,  Instructor,  Recent  Trends,  Specialized 
Literature 

54.  Ragan,  L.  C.  (1998),  Good  Teaching  is  Good  Teaching:  An  Emerging  Set  of 
Guiding  Principles  and  Practices  for  the  Design  and  Development  of  Distance 
Education,  DEOSNEWS ,  5(12),  8. 

A  project  started  in  1995  with  a  grant  from  the  AT&T  Foundation,  entitled  the 
Innovations  in  Distance  Education  (IDE)  Project,  led  to  the  development  of  a  set  of 


54 


guiding  principles  for  the  design  and  development  of  distance  education  programs.  The 
resulting  principles  and  practices  also  apply  equally  well  to  the  resident  instruction  as 
well  as  the  distance  education  delivery  model.  "Good  teaching  is  good  teaching"  and 
while  the  techniques  may  change  depending  on  the  constraints  of  time  and  place,  the 
desired  student  goals  are  basically  the  same.  The  principles  deal  primarily  with  the 
design  and  development  stages  of  the  instructional  design  process. 

Keywords:  Instructor,  Recent  Trends,  Specialized  Literature 

55.  Raschke,  C.  (1999),  Telecommunications  and  New  Learning  Paradigms, 
Syllabus,  Nov/Dec  1999,  4  pp. 

This  article  examines  paradigms  of  teaching  and  learning  from  a  historical 
perspective  in  order  to  provide  some  clarity  to  what  the  terms  "teaching"  and  "learning" 
mean  with  respect  to  the  philosophy  of  education. 

The  first  paradigm  of  teaching  and  learning  is  referred  to  as  mandarin,  and  goes 
back  to  the  imperial  courts  of  China.  The  principle  purpose  of  teaching  is  viewed  as  the 
transmission  of  ancient  cultural  norms,  practices,  and  values.  Rote  instruction, 
memorization,  and  mastery  of  select  skills  such  as  reading  and  calligraphy  characterize 
the  mandarin  model. 

The  second  paradigm  is  the  academic,  derived  from  the  ancient  Greek  academy  in 
Athens.  It  is  the  cornerstone  of  the  modern  liberal  arts  education.  It  focuses  on  the 
self-development  of  the  learner  through  the  challenging  and  charismatic  role  model  in 
the  person  of  the  teacher.  The  role  of  teaching  is  to  produce  lifelong  learners  in  the 
academic  model. 

The  third  paradigm  is  the  clerical,  which  reflects  the  historical  role  of  religious 
hierarchies  in  shaping  secular  institutions.  The  goal  of  instruction  in  the  clerical  model 
is  to  inculcate  in  the  student  the  kinds  of  beliefs,  values,  and  moral  practices  that  will 
ensure  a  virtuous  and  productive  life  and  ultimately  eternal  salvation. 

The  fourth  paradigm  is  the  industrial  model,  first  developed  in  Germany  and 
imported  to  the  US  in  the  1800s.  It  includes  the  K-12  schooling,  community  college 
systems,  and  the  land  grant  university.  The  industrial  model  was  based  on  the 
discovery  that  in  a  capitalist  economy,  workers  needed  to  be  increasingly  equipped  with 
scientific  and  technical  know-how. 

The  fifth  and  final  paradigm  discussed  is  the  transactional  model,  or  informational 
model.  Teaching  and  learning  are  no  longer  separate  activities  in  the  generation  of 
knowledge.  The  transactional  paradigm  delimits  a  new  "learning  space,"  by  which  is 
meant  the  total  context,  or  set  of  conditions,  within  which  learning  at  a  particular 
juncture  takes  place.  With  respect  to  the  distributed  learning  environment,  the 
learning  space  is  considerably  different  from  the  traditional  learning  space  of  a  typical 
class  characterized  by  a  teacher  in  front  of  a  group  of  students  who  listened,  took  notes, 
and  asked  questions.  In  the  distributed  learning  environment,  there  is  a  complex  and 
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constantly  shifting  set  of  queries  and  responses  on  the  part  of  both  instructor  and 
student,  rendering  the  distinction  between  ’’educator"  and  "educated"  less  significant. 
With  transactional  learning,  teaching  and  learning  are  no  longer  separate  activities. 

The  instructor  is  no  longer  the  sole  originator  of  all  academic  content  within  the 
learning  space.  In  the  learning  space  of  a  digital  culture,  there  can  be  a  virtual 
explosion  of  "self-crafted,  ad  hoc,  and  customized  learning  modules."  The  digital 
learning  space  is  characterized  by  experimentation  and  exploratory  activity,  spurred  on 
by  the  infinite  web  of  files  and  hyperlinks  that  constitute  the  Internet. 

Keywords:  Course  Design,  Instructor,  Recent  Trends,  Practitioner  Literature 

A  recent  Army  training  experiment  involved  integrating  the  computer 
training  on  the  Army  Battle  Command  System  (ABCS)  with  existing 
tactics  courses  at  CGSC  to  provide  field  grade  officers  with  needed  digital 
systems  training  using  the  same  amount  of  time  previously  devoted  only  to 
tactics  training . 

56.  Ross,  K.,  G.,  Ph.D.,  and  Yoder,  M.  K.  R.  (1999),  Producing  Computer  Literacy 
for  the  Digitized  Battlespace  of  the  Future .  Paper  presented  at  the 
Interservice/Industry  Training,  Simulation  and  Education  Conference, 
Orlando,  FL. 

Computer  literacy  is  a  decisive  future  battlefield  skill.  The  goal  of  a  recent  Army 
training  experiment  at  the  US  Army  Command  and  General  Staff  College  (CGSC)  was 
to  integrate  tactics  education  with  training  on  the  Army  Battle  Command  System 
(ABCS)  in  a  brigade  operations  course  to  produce  both  computer  performance  skills  and 
tactical  competency  using  the  same  amount  of  time  previously  devoted  to  tactics 
training  alone.  The  methodology  consisted  of  the  radical  redesign  of  an  existing  tactics 
course,  including  the  integration  of  a  previously  separate  40-hour  computer  operator 
course.  The  new  course  design  was  based  on  the  constructivist  approach  to  instruction 
which  places  the  learner  in  an  active  role,  not  just  a  passive  receiver  of  information,  but 
engaging,  seeking  to  make  sense  of  things,  interpreting  experiences,  and  constructing 
guides  to  how  knowledge  can  be  used.  The  instructor  intervenes  only  when  the  students 
reach  a  point  of  no  progress.  The  after  action  review  (AAR)  is  a  key  component  of  the 
approach.  Instructors  must  be  thoroughly  versed  on  the  learning  objectives  as  well  as 
the  enabling  tasks  that  the  students  must  perform  to  meet  those  objectives  and  must  be 
skilled  facilitators  in  both  providing  feedback  and  in  conducting  the  AARs.  Instructors 
must  also  be  thoroughly  proficient  on  all  the  tools,  including  tactical  computers.  The 
classroom  environment  must  be  as  close  to  the  environment  that  the  task  must  be 
transferred  to  as  is  practical. 

Results  of  the  experiment  were  uniformly  positive.  The  students  consistently 
achieved  all  learning  objectives,  and  with  no  additional  training  time,  learned  to  operate 
and  apply  the  digital  systems.  Students  that  were  assigned  to  jobs  where  they  use  the 
MCS  reported  high  retention  of  the  skills  learned  in  the  classes.  Researchers  assigned 
this  largely  to  the  contextual  basis  of  the  original  learning.  The  results  indicated  that 
even  students  who  came  to  the  course  with  minimal  understanding  of  tactics  and/or  the 
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digitized  tactics  systems  could  produce  high  quality  operational  products  using  digitized 
systems  from  the  147  hours  of  training. 

Keywords:  Course  Design,  Automation,  Specialized  Literature 

57.  Russell,  T.  L.  (1999),  The  No  Significant  Difference  Phenomenon,  Chapel 
Hill,  NC,  Office  of  Instructional  Telecommunications,  North  Carolina 
University. 

In  this  book,  Russell  offers  evidence  to  support  the  position  that  distance  education 
programs  match  conventional  courses  in  rigor  and  quality  of  outcomes.  The  evidence 
comes  in  the  form  of  brief  annotations  of  over  200  citations  illustrating  the  "no 
significant  difference"  phenomenon.  There  is  a  Web  site  at  the  University  of  New 
Brunswick  to  provide  current  updates  of  citations  and  readers  can  submit  their  entries 
for  posting:  http://tenb.mta.ca/nhenom/. 

Keywords:  Benefits,  Specialized  Literature 

58.  Sachs,  D.  D.  (2000,  17-20  September  2000),  Pace  University  NACTEL 
Program.  Paper  presented  at  the  ALN  Workshop  2000:  Learning 
Effectiveness,  Faculty  Satisfaction,  and  Cost  Effectiveness,  Lake  George,  NY. 

Keywords:  Costs,  Specialized  Literature 

The  NACTEL  program  consists  of  an  Associates  of  Science  Degree  (AS)  in 
Telecommunications  that  is  provided  100  percent  over  the  Internet  to  individuals  who 
are  currently  employed  by  telecommunications  companies  such  as  Bell  Atlantic  and 
General  Telephone  and  Electronics  (GTE),  and/or  who  belong  to  two 
telecommunications  unions.  The  NACTEL  Program  is  administered  by  Pace 
University,  a  private  university  located  in  the  NY  metropolitan  area.  The  full  program 
was  made  available  in  fall  1999  and  consists  of  16  courses,  half  of  which  are  in  general 
education  and  half  of  which  are  in  electronics  and  telecommunications.  The  program  is 
focused  on  providing  more  network  technicians  to  the  companies,  and  most  individuals 
work  full-time  while  obtaining  their  education.  Often  their  tuition  and  fees  are 
reimbursed  by  the  companies.  The  average  tuition  cost  per  course  is  $500,  considerably 
less  than  the  tuition  charged  to  the  traditional  students  attending  this  private  NY  state 
university. 

Currently,  the  NACTEL  project  expenses  and  revenue  are  just  about  even.  Start-up 
costs  for  this  program  were  $500,000,  which  was  provided  by  the  Alfred  P.  Sloan 
Foundation.  Of  the  initial  $500,000,  half  went  to  course  development  and  half  went  to 
overall  management  and  organization  expenses.  Faculty  members  develop  the  course 
content,  select  the  textbooks,  and  create  videotapes  with  technical  support  from  the 
School  of  Computer  Science  and  Information  Systems  at  Pace  University.  The  budget 
for  online  course  development  is  $15,000  per  course.  There  were  many  unanticipated 
expenses  when  the  program  first  began,  and  for  the  first  year,  revenue  and  expenses 
were  at  a  slightly  less  than  break-even  point.  At  the  anticipated  student  enrollment 
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levels  for  the  2000-2001  year  (240  students  times  5  enrollment  sessions),  revenue  and 
expenses  are  expected  to  break-even.  The  program  is  expected  to  generate  profits  in  the 
short  term. 

The  benefits  of  the  NACTEL  program  include  a  wider  student  base  to  include  adults 
that  work  full-time  in  37  different  states  who  would  otherwise  not  be  able  to  attend 
school.  Statistics  from  the  first  year  of  the  program  indicate  that  students  are 
succeeding  in  the  ALN  classes  they  are  taking.  The  program  has  also  introduced  a  large 
number  of  faculty  members  to  online  education  technology  and  has  provided  faculty 
with  leading  edge  hardware  and  software  that  is  shared  with  other  university  members. 
The  program  has  led  to  the  development  of  several  other  online  initiatives,  which  are 
already  profitable  because  they  charge  full  tuition.  Overall,  the  benefits  have  surpassed 
the  costs. 


The  Department  of  Energy  began  sponsoring  a  DL  project  in  the  early 
1990s  to  expand  the  work-pool  of  trained  waste- management  workers  for 
environmental  clean-up  projects. 

59.  Stapp,  M.  (1999),  New  Mexico  State  University  (NMSU)  and  the  Waste- 
management  Education  and  Research  Consortium  (WERC),  Personal 
Interview,  Las  Cruces,  NM. 

NMSU  and  WERC  have  produced  interactive  television  (ITV)  courses  between  the 
Consortium  members  and  transmitted  ITV  courses  across  the  state  and  nation, 
including  some  internationally  transmitted  courses,  since  1990.  WERC  was  established 
through  a  Department  of  Energy  sponsorship  to  expand  the  work-pool  for  waste 
management  cleanup. 

Professors  often  must  transfer  hand  written  notes  to  PowerPoint  presentations  for 
the  televised  media.  Chalkboards  are  also  passe,  as  they  are  hard  to  read  for  the 
distant  students.  The  preparation  time  is  a  good  4-6  times  that  of  a  traditional  class, 
according  to  the  production  staff  on  campus.  They  try  to  get  the  professors  up  to  speed 
as  early  as  possible  on  the  specifics  of  televised  teaching,  the  cameras,  sound,  student 
interactions,  etc.  Motivation  is  key:  the  professor  must  want  to  teach  a  televised  course 
in  order  for  it  to  be  a  positive  experience  for  all  participants.  Student  feedback  is  used 
for  course  evaluation;  if  the  students  have  negative  feedback,  then  that  professor  is 
generally  not  invited  back  to  teach  again.  Some  professors  have  a  difficult  time  in  front 
of  the  television  and  others  are  naturals.  Two-way  interactive  video  is  also  used  for 
some  courses,  and  Web-based  courses  are  being  developed.  Many  professors  are 
designing  Web-components  for  assignments,  and  interaction  with  the  students  to 
augment  the  televised  courses.  Professors  that  teach  the  WERC  courses  are 
compensated  an  additional  $1,000  per  semester  and  are  provided  $2,500  to  hire 
graduate  assistants  to  help  them  with  the  course  load.  They  are  also  provided  with  a 
$200  copy  budget.  Courier  services  are  used  to  get  handouts  to  the  distant  students.  A 
site  facilitator  is  responsible  for  getting  the  handouts  to  the  students.  Facilitators  also 
open  up  the  classroom,  start  up  the  equipment,  etc.  The  lag  time  for  the  distant  student 
to  receive  and  then  send  homework  assignments  makes  scheduling  a  crucial  factor.  A 
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tape  library  is  available  for  students  to  check  out  tapes  at  the  library  and  at  the 
production  facilities  on  campus. 

Keywords:  Benefits,  Instructor,  Specialized  Literature 

60.  Tapscott,  D.  (1999),  Growing  Up  Digital:  The  Rise  of  the  Net  Generation, 
McGraw-Hill  Professional  Publishing. 

Growing  Up  Digital  explores  the  differences  evident  between  the  net  generation, 
born  since  1977,  and  the  baby  boomer  generation,  those  born  between  1946  and  1964. 
The  emphasis  is  on  the  impact  of  the  Internet  and  the  new  interactive  technologies  on 
our  children.  Despite  fears  that  the  computer  is  making  kids  more  antisocial,  the 
author  indicates  that  kids  are  using  the  computer  to  play,  build  relationships,  and 
explore  the  world. 

One  section  of  the  book  explores  the  "New  Teacher"  as  the  digital  media  enter  the 
schools.  According  to  Tapscott,  teachers  need  to  become  more  youth  workers,  not 
merely  transmitters  of  information.  Teachers  can  help  students  navigate  through  all 
the  information  and  provide  crucial  guidance  and  support  regarding  how  to  learn.  The 
teacher  mediates  the  students'  engagement  with  the  Internet,  ensuring  the  Internet  to 
be  stimulating  and  intelligent.  To  do  this,  teachers  must  become  as  fluent  in  new  media 
as  their  students.  Teachers  are  becoming  facilitators  in  creating  citizens  in  a  global 
society.  Students  are  becoming  in  charge  of  their  own  learning;  only  they  can  decide  to 
do  something  and  what  they  decide  to  do  will  be  far  more  creative  than  you  can  imagine. 

Keywords:  Benefits,  Instructor,  Specialized  Literature 

61.  Tapscott,  D.  and  Caston,  A.  (1993),  Paradigm  Shift:  The  New  Promise  of 
Information  Technology,  NY,  McGraw-Hill,  Inc. 

The  information  age  is  evolving  into  a  second  era,  driven  by  profound  changes  in  the 
nature  of  computers  and  by  the  demands  of  competitive  business.  A  paradigm  shift  in 
information  technology  is  creating  a  new  business  environment.  A  number  of  recurring 
business  themes  are  emerging  as  companies  reengineer  themselves  for  the  new 
environment:  productivity  of  knowledge  and  service  workers,  quality,  responsiveness, 
globalization,  outsourcing,  and  partnering. 

Keywords:  Benefits,  Recent  Trends,  Specialized  Literature 

62.  Throne,  M.  H.,  and  Lickteig,  C.  W.  (1997),  Training  Computer  Skills  for  the 
Future  Battlefield:  A  Review  and  Annotated  Bibliography  (Research  Product 
97-15),  Alexandria,  VA,  US  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences. 

The  Army  conducted  several  AWEs  in  the  1990s  to  test  the  networked  digital 
information  systems  and  corresponding  battlefield  scenarios  for  the  digital  battlefield. 
One  of  the  first  AWEs,  Desert  Hammer  VI,  concluded  that  the  computer-based 
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information  systems  could  not  replace  fundamental  combat  skills.  Rather,  a  three- 
phase  hierarchical  approach  to  training  was  recommended:  fundamental  combat  skills, 
computer  skills,  and  finally,  the  integration  of  combat  and  computer  skills  in  a 
warfighting  environment.  Follow-on  AWEs  found  that  digital  skills  are  also  highly 
perishable. 

This  report  contains  an  annotated  bibliography  of  76  articles  on  programming, 
software,  simulations,  and  gaming  skill  acquisition,  retention,  and  transfer.  General 
findings  for  skill  acquisition  include:  procedural  job  skills  should  be  acquired  before 
supporting  computer  skills,  novices  apply  their  mental  models  to  computer  skills 
sometimes  erroneously,  guided  exploration  improves  computer  skill  acquisition  more 
than  just  reading  a  manual,  active  participation  increases  learning,  novices  may  acquire 
skills  faster  with  direct  manipulation  or  windows-based  applications  than  with 
command-based  systems,  and  advance  organizers  may  improve  computer  skill 
acquisition. 

With  respect  to  retention,  over-practice  increases  retention  of  computer  skills, 
problem-solving  strategies  may  slow  acquisition  but  increase  retention,  process 
knowledge  about  how  an  application  operates  is  retained  better  than  the  interface-based 
operating  procedures,  and  commands  with  more  specific  names  are  retained  better  than 
commands  with  more  generic  names.  Computer-assisted  instruction  (CAI)  may  actually 
lead  to  greater  transfer  of  computer  skills  than  actual  system  training  and  is  often 
cheaper  than  simulation. 

Keywords:  Benefits,  Automation,  Specialized  Literature 

63.  Wentling,  T.  L.  (2000,  17-20  September  2000),  Cost  Efficiency  of  Online 
Instruction  in  a  Research  One  University:  A  Case  Study  of  a  Department's 
Efforts .  Paper  presented  at  the  ALN  Workshop  2000:  Learning  Effectiveness, 
Faculty  Satisfaction,  and  Cost  Effectiveness,  Lake  George,  NY. 

This  article  describes  the  online  Masters  program  entitled  "Global  Human  Resource 
Development"  offered  by  the  University  of  Illinois  beginning  in  the  fall  of  1998.  The 
same  degree  is  also  offered  in  the  traditional  campus.  The  target  audience  for  the 
program  includes  individuals  who  are  working  in  the  private  sector  or  government 
sector  in  such  positions  as  instructional  designer,  trainer,  training  evaluator, 
performance  consultants,  etc.  The  nine-course  degree  program  can  be  completed  by 
students  who  hold  full-time  jobs  in  3  years,  at  one  course  per  semester.  The  program 
relies  primarily  on  Internet-based  technologies  that  are  augmented  with  CD-ROM  and 
telephone  in  some  courses. 

The  Human  Resources  Department,  which  initiated  this  program,  has  a  fairly 
sophisticated  faculty  in  terms  of  technology  use,  with  faculty  using  Web  support  for 
most  on-campus  courses.  The  Sloan  Foundation  support  has  been  a  major  driver  of 
technology  infusion  in  course  delivery  at  the  University  of  Illinois.  All  online  students 
are  charged  the  in-state  tuition  rate  regardless  of  residence,  which  translates  to  quite  a 
bargain. 
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Program  start-up  costs  were  quite  high.  The  university  provided  $180,000  for  the 
nine-course  program  and  the  department  invested  another  $76,000.  The  university 
plans  to  run  three  groups  with  an  average  of  30  students  each  simultaneously  at  all 
times.  A  study  of  learning  outcomes  and  satisfaction  was  completed  during  the  first 
year  of  the  program  that  confirmed  comparable  learning  with  on-campus  instruction 
(Aragon  et  al.,  in  press). 

The  advent  of  online  Masters'  programs  has  resulted  in  changes  in  the  university 
budgeting  process.  The  departments  have  some  latitude  with  respect  to  online  tuition 
decisions  and  also  can  negotiate  the  return  of  the  majority  of  the  online  revenue  back  to 
the  department  developing  the  program,  as  opposed  to  the  university  overall.  This 
decentralization  provides  financial  incentives  for  program  success.  The  university  is 
also  conducting  studies  on  faculty  compensation  issues. 

This  online  program  has  also  developed  an  evaluation  system  to  continuously 
monitor  the  health  of  the  online  program  using  six  vital  signs:  program  demand, 
student  satisfaction,  faculty  satisfaction,  retention,  student  learning,  and  financial 
efficiency.  If  a  substandard  reading  is  obtained  for  any  vital  sign,  an  in-depth  analysis 
is  conducted,  and  a  plan  of  action  for  correcting  the  problem  is  generated.  Cost  drivers 
for  the  online  program  include  fixed  costs  such  as  administrative  time,  secretarial 
support,  equipment,  facility,  server  support,  network  support,  and  office  communication 
costs.  Variable  costs  include  faculty  salary,  technology  support  staff,  teaching  assistant 
salary,  course  materials,  and  mailing  costs.  The  program  achieved  break-even  status 
even  with  less  than  the  targeted  30  students  per  course.  However,  the  cost  analysis 
results  emphasize  the  importance  of  recruiting  students  and  maintaining  enrollments 
in  courses.  A  loss  of  three  students  can  mean  a  loss  in  revenue  of  $2,700.  Currently 
studies  are  underway  to  determine  the  optimal  enrollment  level,  and  the  optimal  level 
of  teaching  and  research  assistant  support  necessary  to  support  the  courses. 

Keywords:  Costs,  Specialized  Literature 

64.  West,  F.  J.  (1999,  August  1999),  Information  Technology:  Advice  from 
Silicon  Valley  CEOs,  Marine  Corps  Gazette. 

During  a  recent  Marine  Corps  AWE,  a  strategic  wargame  was  conducted  at  Stanford 
Business  College  that  paired  10  generals  and  5  admirals  with  15  top  business  CEOs 
from  Silicon  Valley.  The  primary  purpose  of  the  wargame  was  for  the  military  to  obtain 
the  advice  and  business  perspective  of  the  leading  commercial  IT  corporations  with 
respect  to  information  technology  investments.  Advice  from  the  business  executives 
included  reexamining  the  disparity  between  IT  investment  in  garrison  (41  percent  of 
total  budget)  versus  the  tactical  level  (17  percent)  and  investing  more  into  internal 
Marine  training  to  advance  computer  literacy,  adding  that  they  believed  the  military 
should  establish  the  goal  of  ensuring  that  the  entire  force  be  computer  literate  by  2010. 
The  CEOs  suggested  that  the  Marines  put  investment  money  into  two  common 
infrastructures:  wire  systems  for  garrison  and  wireless  systems  for  expeditionary 
warfare.  It  is  premature  to  select  any  one  system  and  deploy  it  across  the  force,  advised 
the  executive  officers,  who  urged  that  the  Marines  not  wait  for  one  model  to  fit  all;  that 


61 


path  will  always  put  the  military  behind.  After  introducing  different  types  of  field 
devices  and  prototype  systems,  the  military  should  let  change  happen,  according  to  the 
CEOs,  adding  that  this  is  the  way  that  industry  approaches  decisions. 

Keywords:  Automation,  Practitioner  Literature 

65.  Westfall,  P.  J.  L.  (1998,  21-22  January  1998),  Distance  Learning:  A  Strategic 
Learning  Initiative  for  the  Air  Force,  Paper  presented  at  the  Government 
Learning  Technology  Symposium. 

This  DL  overview  by  the  creator  of  the  Center  for  Distance  Education  for  the  Air 
Force  Institute  of  Technology  covers  a  number  of  topics  pertinent  to  the  development  of 
DL  programs,  from  the  need  to  understand  the  technologies  to  the  factors  likely  to 
impact  learning  outcomes.  Dr.  Westfall  points  out  that  there  is  no  significant  difference 
in  learning  outcomes  if  appropriate  media  are  used.  He  cautions  that  one  should  be 
skeptical  of  claims  that  students  learn  50  or  60  percent  more  with  one  medium  than 
with  another.  The  outcome  of  the  learning  is  largely  based  on  prior  knowledge  and 
intrinsic  motivation.  Research  conducted  by  the  author  has  shown  a  .78  correlation 
with  these  two  variables  with  the  ability  to  learn.  Do  not  expect  dramatic 
improvements  in  learning  outcomes  because  of  the  media  chosen  to  deliver  the  training. 
Media  choices  can  affect  the  time  the  learning  takes,  however.  For  example,  CBI  allows 
the  student  to  go  right  to  the  material  he  or  she  needs  to  learn,  which  can  shorten 
learning  time. 

Keywords:  Benefits,  Specialized  Literature 

Currently ,  the  Army  operates  an  extensive  infrastructure  of  training 
institutions  that  utilize  considerable  manpower ,  equipment ,  and  facilities. 

These  US  Army  schools  have  faced  especially  intense  budget  pressure  in 
the  decade  of  the  90s.  To  cope  with  these  challenges ,  the  Army  is  devising 
new  training  strategies  that  could  achieve  similar  effectiveness  to  that  of 
current  methods  but  with  substantial  resource  savings. 

66.  Winkler,  J.  D.,  Kirin,  S.  J.,  and  Uebersax,  J.  S.  (1992),  Linking  Future 
Training  Concepts  to  Army  Individual  Training  Programs  (R-4228-A),  Santa 
Monica,  CA,  RAND. 

A  RAND  study  performed  in  the  early  1990s  identified  potential  MOSs  in  which 
traditional  methods  of  individual  training,  with  special  attention  to  training  conducted 
in  residence  at  Army  schools,  could  be  exchanged  for  alternative  training  strategies, 
including  DL,  training  devices,  and  civilian  training  of  military  skills.  The  research 
approach  was  to  identify  first  those  MOSs  in  which  substantial  resources  could  be  saved 
by  implementing  new  training  concepts  for  initial,  entry-level  training  of  enlisted 
personnel.  Training-related  characteristics  of  entry-level  enlisted  MOSs  were 
summarized  by  a  small  number  of  general  dimensions,  the  most  important  of  which 
were  ability  requirements,  civilian  exchangeability,  dominant  tasks,  and  cost  to  train. 
Ability  requirements  indicate  the  degree  to  which  the  MOSs  requires  general 
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intelligence,  specific  vocational  aptitude,  and  pre-service  education.  Civilian 
exchangeability  indicates  the  similarity  between  Army  MOSs  and  civilian  jobs  and 
training  programs.  Dominant  tasks  indicate  whether  the  duties  of  the  MOSs  emphasize 
cognitive  or  informational  tasks,  as  opposed  to  procedural  or  manipulative  tasks.  The 
cost  dimension  indicates  the  MOSs  with  significantly  higher  training  costs. 

The  study  then  provides  an  analytic  framework  for  using  the  analysis  of  training- 
related  characteristics  of  Army  MOS  to  identify  specific  MOS  in  which  significant  cost 
savings  might  be  achieved  by  implementing  one  of  three  alternative  training  concepts: 
(1)  distributed  training,  which  is  the  applied  use  of  DL  technologies  to  support  training 
outside  the  schoolhouse,  at  home-stations,  regional  training  centers,  and  other  selected 
sites,  (2)  training  devices,  simulators,  and  simulations,  and  (3)  substitution  of  civilian 
training  or  job  experience. 

Keywords:  Benefits,  Costs,  Automation,  Specialized  Literature 

67.  Wisher,  R.  A.,  Champagne,  M.  V.,  Pauluk,  J.  L.,  Eaton,  A.,  Thornton,  D.  M., 
and  Curnow,  C.  K.  (1999),  Training  Through  Distance  Learning:  An 
Assessment  of  Research  Findings  (Technical  Report  1095),  US  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences. 

This  review  of  the  DL  literature  was  conducted  to  support  the  ARI  TRAINTODAY 
project,  the  goal  of  which  is  to  determine  whether  it  is  better  to  train  up-front  in  the 
classroom  or  just  in  time  in  the  field.  Searches  on  relevant  research  databases  resulted 
in  2000  entries,  largely  related  to  education.  Of  these,  43  research  reports  were 
considered  most  relevant  to  the  subjects  of  experimental  design,  course  conversion,  or 
training  applications.  The  studies  analyzed  in  this  review,  with  a  combined  sample  size 
of  5,438  students,  presented  a  generally  positive  view  of  the  effectiveness  of  DL  for 
training  applications.  However,  when  the  authors  tried  to  determine  why  DL  was 
effective,  they  found  that  40  percent  of  the  reports  included  in  this  summary  did  not 
mention  course  conversion  or  redesign,  25  percent  did  not  mention  instructional 
techniques,  and  50  percent  of  reports  did  not  fully  describe  the  method  of  technology 
employed.  Video  teletraining  was  the  dominant  technology  reported.  Comparison 
groups  were  available  only  one-third  of  the  time.  Thus,  some  of  the  findings  from  the 
reports  reviewed  were  not  supported  by  a  strong  experimental  design. 

Keywords:  Benefits,  Specialized  Literature 

68.  Wisher,  R.  A.,  and  Priest,  A.  N.  (1998),  Cost-effectiveness  of  Audio 
Teletraining  for  the  US  Army  National  Guard,  The  American  Journal  of 
Distance  Education,  12(1). 

This  article  summarizes  a  cost-effectiveness  analysis  of  training  Army  National 
Guard  soldiers  by  audio  teletraining  technology.  Two  hundred  and  twenty-five  soldiers 
attending  the  Unit  Clerk  Course  in  1996  were  divided  into  two  groups;  107  soldiers 
participated  in  the  resident  classroom  version  of  the  Unit  Clerk  Course  while  1 18 
participated  in  the  audio  teletraining  version.  The  resident  course  is  conducted  at 
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Camp  Robinson,  AR  and  required  that  soldiers  travel  to  Camp  Robinson,  be  billeted, 
and  provided  meals.  Both  groups  used  the  same  student  workbooks,  job  aids,  and  tests. 
The  audio  teletraining  version  required  no  additional  development  costs  to  reconfigure 
this  course  to  a  DL  format.  The  audio  teletraining  format  lacks  the  face-to-face  delivery 
that  other  more  technologically  sophisticated  distance  learning  methods  such  as 
CD-ROM,  videoconferencing,  and  multimedia  packages  can  offer,  but  is  felt  can  be  well 
suited  for  instruction  in  which  static  visual  information  can  be  provided  to  students  in 
advance  and  for  training  tasks  in  which  the  student  follows  directions  from  an 
instructor,  such  as  procedural’tasks. 

The  authors  emphasize  the  importance  of  understanding  the  task  at  hand  and  then 
choosing  the  best  tool  for  it.  Acknowledging  that  there  are  more  expensive  ways  to 
accomplish  this  training,  they  make  a  case  for  the  learning  effectiveness  of  audio 
teletraining  in  comparison  to  face-to-face  instruction  before  turning  to  its  cost- 
effectiveness. 

Keywords:  Costs,  Specialized  Literature 

The  role  of  DL  for  purposes  of  keeping  military  skills  sharp  by 
providing  refresher  training  delivered  to  soldiers  in  the  field ,  as  well  as 
providing  training  to  national  guard  and,  reserve  soldiers  has  enormous 
potential .  The  following  report  provides  a  summary  of  the  studies 
performed,  on  military  skill  retention. 

69.  Wisher,  R.  A.,  and  Sabol,  M.  A.  (1999),  Staying  Sharp:  Retention  of  Military 
Knowledge  and  Skills  (Special  Report  39),  US  Army  Research  Institute. 

This  report  summarizes  research  performed  over  the  past  25  years  on  the  topic  of 
skill  retention,  with  an  emphasis  on  research  performed  by  ARI,  but  also  including 
relevant  research  by  other  military  and  academic  laboratories. 

The  Different  Kinds  of  Skills.  Knowledge,  decision,  and  execution  are  three  skills 
present  in  most  tasks.  In  procedural  tasks,  the  crucial  recall  required  is  memory  for  the 
steps  to  be  performed  in  a  given  procedure.  In  many  military  tasks,  knowledge  retrieval 
predominates.  The  second  ability,  sometimes  called  cognitive  processing,  dominates  in 
tasks  such  as  decision-making  by  officers  and  troubleshooting  of  faulty  equipment.  In 
perceptual-motor  tasks,  precise  execution  of  well-practiced  actions  is  the  crucial  aspect. 

Recent  work  in  the  neuroscience  of  human  memory  indicates  that  these  three  types 
of  abilities  are  located  and  controlled  in  different  regions  of  the  brain,  thus,  it  is  not 
surprising  that  studies  have  shown  a  different  pattern  of  forgetting  for  each  type. 
Research  on  memory  for  cognitive  tasks,  such  as  problem  solving,  judgment,  and 
analysis  leading  to  a  decision,  shows  a  moderate  rate  of  decay;  forgetting  occurs  but  is 
relatively  small  for  up  to  a  year  after  learning.  A  summary  of  more  than  20  studies, 
including  two  studies  of  military  tasks  found  an  average  retention  loss  of  20  percent  13 
weeks  after  training.  Memory  for  job  knowledge,  facts  and  ideas  that  a  soldier  must 
remember  in  order  to  perform  successfully,  is  sometimes  referred  to  as  school 
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knowledge.  Studies  of  memory  for  knowledge  learned  in  school  have  sometimes  found 
remarkable  resistance  to  forgetting  for  years  after  learning.  However,  there  is  a  critical 
distinction  between  recognition  memory  and  recall  memory.  Recognition  memory 
involves  choosing  among  provided  alternatives  and  is  usually  tested  with  multiple 
choice,  matching,  or  true-false  test  items.  Recall  memory  requires  the  learner  to 
produce  information  without  being  presented  with  alternatives  and  is  usually  tested 
with  short-answer,  fill-in,  or  essay  test  items.  In  general,  recognition  memory  is 
superior  to  recall  memory.  Data  from  40  studies  of  memory  for  school  knowledge 
indicate  that  recall  memory  retention  is  only  around  55  percent  26  weeks  after  training 
compared  to  recognition  memory  retention  of  close  to  80  percent  26  weeks  after  training. 

Keywords:  Benefits,  Automation,  Recent  Trends,  Specialized  Literature 

70.  Young,  J.  R.  (2001),  Four  Universities  Create  an  Alliance  Against 
Onslaught  of  Technology  Vendors,  Chronicle  of  Higher  Education  (Daily  News 
1/23/2001),  3. 

Four  universities,  University  of  California  Berkeley  extension  program, 
Pennsylvania  State  University's  World  Campus,  the  University  of  Washington,  and  the 
University  of  Wisconsin's  Learning  Innovations  program,  have  formed  an  alliance  to 
share  information  and  make  joint  technology  purchases. 

The  four  universities  have  much  in  common  when  it  comes  to  distance  education, 
each  offering  more  than  100  online  courses,  and  all  being  active  in  earlier  forms  of 
distance  education,  like  correspondence  courses.  The  alliance  will  meet  two  or  three 
times  a  year  and  participate  in  periodic  conference  calls.  The  agenda  will  focus  on  four 
key  areas  --  marketing,  technical  issues,  student  services,  and  faculty  relations. 

Keywords:  Benefits,  Practitioner  Literature 
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